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ABSTRACT 


The Department of Defense’s proposed Range Rule greatly increases the number 
of unexploded ordnance (UXO) contaminated sites that the services must decontaminate. 
Existing models for estimating the cost of UXO removal often require a high level of 
expertise and provide only a point estimate for the costs; they do not provide a 
probability distribution of the potential costs. This thesis presents a probabilistic cost 
estimation model created as an “add-in” for Microsoft Excel. A test database consisting 
of descriptive and cost information on the historic cleanup of nineteen contaminated areas 
is created. To demonstrate the model, the thesis filters the database to find eight historic 
records characteristically similar to a fictitious cleanup scenario, and uses information 
from these historic records to build probability distributions for six cost elements. The 
model applies Monte Carlo simulation to these probability distributions to build a 
probability distribution for the total cleanup cost. The resulting distribution shows that 
for this cleanup scenario the most likely per acre cost is $8,400, but there is a 75% chance 
that costs fall between $8,500 and $26,000. Results for a scenario composed of three 
cleanups predicts a most likely total cost of $1.7 million with a 50% probability of costs 


falling between $1.7 million and $2.2 million. 
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DISCLAIMER 


The reader is cautioned that computer programs developed in this research may 
not have been exercised for all cases of interest. While every effort has been made, 
within the time available, to ensure that the programs are free of computational and logic 
errors, they cannot be considered validated. Any application of these programs without 


additional verification is at the risk of the user. 
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EXECUTIVE SUMMARY 


The Department of Defense’s proposed Range Rule (CFR 32 Part 178) outlines. 
the military’s environmental response actions for non-active ranges contaminated with 
unexploded ordnance (UXQ). The proposed rule greatly increases the number of UXO 
contaminated areas that the military services must decontaminate. The services need to 
plan the environmental remediation (cleanup) of their contaminated sites in response to | 
the Range Rule, but most of the military services have limited experience in large-scale 
environmental restoration. The existing cost estimating models for ordnance removal 
are engineering cost models that are based on level-of-effort and material costs. These’ 
models often require an expert’s judgement for the inputs and validation of the outputs. 
Furthermore, the current models provide only a point estimate for the costs; they do not 
provide a probability distribution of the potential costs. Headquarters Marine Corps 
(HQMC) desires a method to estimate cleanup costs for Marine Corps sites affected by 
the Range Rule, and an alternative to these existing models. 

This thesis presents a probabilistic cost estimation model that allows for cost-risk 
analysis. It creates a test database from nineteen prior cleanups that are under United 
States Army Corps of Engineers’ management. The database contains detailed physical 
descriptions of the contaminated areas and associated cost information. The database 
segregates cost information by labor costs, material costs, and travel/per Diem costs for 
the Removal Action (RA) and the Environmental Evaluation/Cost Analysis (EE/CA) 
phases of cleanup. This cost structure establishes six cost elements that account for all 


costs in these final two phases of a cleanup. 


X1X 


The model’s design, as an “add-in” for Microsoft Excel, makes it accessible to 
most military users and provides a user-friendly means to filter records and find a pool of 
historic sites that are characteristically similar to a proposed cleanup. Prior research and 
analysis of the data shows that environmental characteristics of the contaminated area, 
characteristics of the UXO, proposed future use, and cleanup methodology, determine 
cleanup costs. Once the user finds an appropriate pool of data using these predictive 
factors, the model assists the user in analyzing the historic cost information and building 
probability distributions for the six cost elements. The model uses a weighted 
distribution for each cost element that is a linear combination of the empirical data and a 
user supplied triangular distribution. Monte Carlo simulation sums the weighted 
probability distributions of the < cost elements in order to build a total cost probability 
distribution for the RA phase and the EE/CA phase of a site restoration. 

A fictitious cleanup scenario demonstrates the model’s functionality. The 
resulting total cost probability distribution reveals that for this cleanup scenario the most . 
likely cost is $8,400 per/acre, but the distribution also shows that there is roughly a 75% 
chance that costs actually fall between $8,500 and $26,000. The thesis further 
demonstrates how Monte Carlo Simulation can aggregate the total cost estimates for 
several related areas to produce a single multi-area estimate. Results for a scenario ~ 
composed of three contaminated areas predict a most likely cost of $1.7 million with a 
50% probability of costs falling between $1.7million and $2.2million. 

This work recommends that future requirements standardize and automate the 


reporting of cost information by contractors. The limited amount of available cost data 


XX 














reduces confidence in the estimates that this model produces. More cost data can provide 
the user with a larger pool of similar cleanups and can reduce the assumptions that the 
user must make in performing an estimate. This thesis aise recommends that the 
Department of Defense consider implementing cost-risk analysis in other models that it 
uses for estimating UXO cleanup costs. 

Although HQMC is the impetus for the development of this model, the cost-risk 
model is suitable for all Department of Defense users interested in producing cost 
estimates for UXO cleanup. The thesis provides an nen model that allows the 
Marine Corps and other users to budget for the cleanup of UXO. Estimating the costs of 
environmental restoration of ordnance contaminated sites is a difficult process containing 
a substantial amount of unknowns. The cost-risk model provides the means to manage 


this uncertainty and risk: 
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I. INTRODUCTION 


The Department of Defense’s proposed Range Rule (CFR 32 Part 178) outlines 
the military’s environmental response actions for non-active ranges contaminated with 
unexploded ordnance (UXO). By publishing an acceptable rule that governs the handling 
of military munitions on closed, transferred and transferring ranges, the DOD hopes to 
avoid having to treat UXO as solid hazardous waste. The proposed rule, however, 
greatly increases the required number of UXO contaminated areas that the military 
services must clean. The services need to plan the environmental remediation (cleanup) 
of their contaminated sites in response to the draft rule, but with the exception of the 
Army the military services have limited experience in large-scale environmental 
restoration. The Army Corps of Engineers, through its involvement in the Defense 
Environmental Restoration Program - Formerly Used Defense Sites (DERP-FUDS) and 
Base Realignment and Closure (BRAC), has extensive experience in remediation of UXO 
sites. The existing cost estimating models for prince removal projects are engineering 
cost models that are based on level-of-effort and material costs. These models often 
require an expert’s judgement for the inputs and validation of the outputs. Further, the 
‘ current models provide only a point estimate for the costs; they do not provide a 
probability distribution of the possible costs. 

This thesis presents a probabilistic cost estimation model that provides the 
military services the ability to perform cost-risk analysis. The user can develop a cost 


estimate using the cost data from historic sites that have characteristics similar to the 





proposed cleanup site. The eigdel produces a probability distribution for the cost of a 
single cleanup or several combined cleanups. These distributions allow a user to budget 
for the cleanup of UXO ona specific site and to make decisions on how to approach the 
remediation of multiple sites. 

Chapter II provides an overview of the UXO problem and current methods of 
estimating UXO cleanup costs. Included in Chapter II is related research in probabilistic 
cost estimating and general cost estimating of environmental projects. Chapter I 
outlines cost factors in the UXO remediation process. Chapter IV presents background 
information necessary for understanding the probabilistic cost model, and this chapter _ 
gives a detailed description of the data used in the model and discusses the general 
method of Monte Carlo Sampling used to produce total cost probability distributions. In 
Chapter V a “walkthrough” of the model is performed using a cleanup scenario to 
illustrate the model's functionality. Discussion of the simulation output and its 


interpretation is given in Chapter VI, and finally Chapter VII delivers recommendations 


and conclusions. 











Ii, BACKGROUND 


A. SCOPE OF THE UXO PROBLEM 


UXO is a massive challenge; worldwide, nearly 500 people a week are killed or 
injured from a staggering 100 million landmines left over from prior conflicts [United 
States Department of Defense, Office of the Under Secretary of Defense for Acquisition | 
and Technology (USD (A&T)) 1998]. Although in the United States death or injury is 
extremely rare, UXO has other substantial detriments. Batinniites show that some level of 
UXO contaminates nearly 15 million acres in the United States. This contamination 
prevents civilian land use, threatens public safety, and causes environmental concern. 
Since many suspect sites have not been surveyed, the extent of this problem is uncertain. 
However, estimates show that if only five percent of these sites require remediation, 
cleanup costs could exceed 15 billion dollars [USD (A&T) 1998}. 

To address the UXO problems, the Environmental Protection Agency (EPA) 
published the Military Munitions Rule in February of 1997. The EPA, however, 
postponed a decision on whether to regulate UXO on closed, transferred, and transferring 
(CTT) ranges while the Defense Department develops its own Range Rule for these CTT 
ranges. If the DOD does not promulgate the Range Rule or if the Range Rule does not 
adequately protect the public and the environment, the EPA plans to force cleanup on 
these CTT ranges under United States environmental law [United States Department of 
Defense, Office of the Secretary 1997, sec. II.1.]. By drafting the Range Rule, the DOD 


hopes to uniquely define UXO from other hazardous wastes, and establish methods to 





handle UXO that is acceptable to the military, EPA, and other stakeholders. The 
different services understand that unless they voluntarily demonstrate responsible 
stewardship of these lands, as addressed in the Range Rule, restrictions could be imposed 
that would severely limit training and create insurmountable costs. 

Due to the implications of the proposed Range Rule, the services are showing 
concern over the many issues surrounding UXO management and removal including the 
analysis of cleanup costs. One reason that the estimation of cleanup costs proves to be 
difficult is that there is tremendous siliect found in the costs from one site to another. 
For example, the budget for the Navy’s Kaho’olawe range in Hawaii is $400 million or 
approximately $14,000 per acre; the Army’s Jefferson Proving Ground estimates are as 
high as $1.4 million dollars per acre; and parcels of Adak, Alaska have estimates up to a 
Staggering $10 million per acre. [Rohrer 1997] 


B. CURRENT STATUS OF UXO COST ESTIMATION 


United States Army Engineering and Support Center Huntsville, Alabama 
USAESCH) manages and has managed the majority of UXO remediation projects in the | 
United States. Throughout the 1970’s and early 1980’s the DERP-FUDS evolved. This 
program’s goal is to remove any risk to human health, safety, and the environment caused 
by former DOD activities. The DOD delegates the execution of this program to the 
United States Army Corps of ee (USACE). As the chief executor of 
environmental activities at FUDS, the USACE is involved in a varying capacity of 


cleanup on approximately 1,500 FUDS. To create a focal point for all UXO activities the 


USACE created a Mandatory Center of Excellence (MCX) at Huntsville, Alabama in 








1990. Along with the DERP-FUDS, the MCX is also responsible for managing UXO 
activities on BRAC sites, and Installation Restoration Program sites. In total the MCX 
has over 2,000 inventoried sites and usually administers from 60 to 80 restoration 
projects at any given time [Foley et al. 1996]. 

Huntsville Center produces cost estimates for UXO projects using either the 
Remedial Action Cost and Engineering Requirements System (RACER) [BTG 
Incorporated (Delta Research Division) 1998], or a proprietary spreadsheet model 
developed by Huntsville Center’s cost engineers. Both RACER and the spreadsheet 
model base calculations on the number of teams that a project requires for UXO removal. 
The two models then determine total labor hours, equipment use, and material 
requirements based on these hypothetical teams. Finally, these two models use labor-rate 
costs and material unit costs to arrive at a single point estimate. Unfortunately, these 
models do not provide any information about cost uncertainty and also require expert 
knowledge of UXO remediation in order to verify inputs and results. The USACE uses 
experienced cost engineers to produce cost estimates. For other agencies these models 


would be difficult to use without background experience in UXO projects. 


Cc. RELATED RESEARCH 


Throughout the DOD cost analysts frequently use cost-risk analysis models, and 
the research in this field is abundant. Anderson and Cherwonik [1997] present an 
overview of cost-risk analysis in military acquisition programs. Their primer describes 
the current DOD shift away from point estimates to probabilistic estimates, and justifies 


the need for this shift. The paper focuses on providing the acquisition Program Manager 








and other integral DOD acquisition personnel with an appreciation of the benefits of cost- 
risk analysis. 

More specifically related to this thesis, the DOD uses cost analysis directly in 
environmental programs. The MITRE Corporation, a subcontractor in the Army’s 
chemical weapons demilitarization program, presents a methodology that compares two 
different processes used for Sania eine chemical weapons [Hughitt, Wusterbath, and 
Zoepfl 1994}. The MITRE Corporation report uses Latin hypercube sampling of work 
breakdown structure cost-elements to create total cost probability distributions for the 
demilitarization process. The report then compares the life cycle costs of the current 
baseline process with the proposed cryofracture process. Since the two processes are in 
different stages of developmental maturity, MITRE develops input probability 
distributions with a combination of theoretical and empirical data. The results of the 
analysis provide a method to compare the economic merits of the two plant designs and 
their associated demilitarization processes. 

In other research, Air Force Materiel Command contracted Aeronautical Systems 
Center to develop a funding estimator tool for pollution prevention in weapon system 
production [Aeronautical Systems Center 1994]. The EPA requirements to reduce the 
use of ozone depleting chemicals forces program managers to develop strategies and 
funding plans to meet the EPA goals. The Air Force Materiel Command model provides 
a means to produce rough order of magnitude funding estimates to support Program 


Objective Memorandum inputs. The model identifies a weapon system’s logistic and 


manufacturing processes and the hazardous material associated with these processes. 











Next, the model assigns each hazardous material involved in the production process an 
importance factor, difficulty factor, and substitute factor. Finally, the model uses 
regression analysis to provide pollution prevention funding estimates. 

Hersey and Pederson’s unpuplished report is one of the earliest works specifically 
addressing UXO removal costs [1981]. Their work outlines the factors that influence 
cleanup costs: characteristics of the contaminated area, characteristics of the ordnance, 
cleanup requirements, and the cleanup methodology. They present an engineering 
“bottom-up” model that provides a means to compare the cost of manual <eniovAl with 
removal assisted by a Sub-Surface Clearance Vehicle. The model compares the costs of 
the two removal methods under varying environmental conditions. 

Delta Research, a division of BTG Incorporated, has done the greatest amount of 
research in estimating costs of UXO removal. Delta Research’s team of environmental 
engineers, cost analysts, contractors, and economists developed the RACER System to 
provide the DOD and the private sector with a system to assist in estimating costs of 
hazardous waste cleanup. RACER consists of over 65 separate engineering cost and 
project management models, with a few dedicated solely to ordnance and explosive waste 
remediation. Delta Research consolidates the UXO models into a separate Microsoft 
Excel [| Microsoft Corporation 1997] based package titled Ordnance senedintion Cost 
Analyzer (ORCA). These models are called RACER/ORCA in the remainder of this 
thesis. These models calculate quantities of equipment, labor, and materials needed for a 
job based on the underlying engineering processes [BTG Incorporated (Delta Research 


Division) 1999]. 














Ill. ORDNANCE REMOVAL PROCESSES AND COST DRIVERS 


A. PHASES OF RESTORATION 
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UXO remediation usually follows three phases: “inventory,” “study,” and 
“removal.” The total cost of site restoration from initial project identification through 
ordnance removal is known as the “cost to complete” (CTC). Estimating the CTC 
requires understanding the costs in each phase of a cleanup. 

1. Inventory Phase 

For a DERP-FUDS project, the inventory phase begins with the development of 
an Initial Project Report (INPR). The INPR determines whether a threat exists, whether 
the site is the responsibility of the DOD, and whether the site is eligible for cleanup under 
DERP-FUDS. The cost for an INPR is usually relatively small (less than $25,000) 
[Fanning 1999]. 

2. Study Phase 

The study phase usually consists of two tasks: the Archive Search Report (ASR) 
and the Engineering Evaluation/Cost Analysis (EE/CA). The ASR is a detailed 
investigation of historic information related to the suspected contaminated areas. The 
agency conducting an ASR reviews any existing paperwork, conducts interviews, reviews 
maps, and visually examines the suspected and surrounding areas. Producing an ASR 
with reliable conclusions can be quite difficult because most military ranges have few if 
any records on their use. Range Boundaries shift over the years, ranges overlay onto 


other ranges, and ranges convert to alternate use. To compound this problem, ranges are 


often not segregated by use and are open to all types of ordnance and ordnance delivery 





[Hersey and Pederson 1981]. The final ASR product includes maps of the suspected 
area, a detailed description of the physical composition of these areas, and a 
determination of the types and densities of the ordnance that may be encountered. The 
Saint Louis district or Rock Island district of the Corps of Engineers normally completes 
ASRs. Regardless of the project’s size, an ASR usually has a fixed price of about 
$40,000 [Fanning 1999]. 

When the ASR identifies areas of contamination, then the study phase conducts 
an EE/CA. The EE/CA is a eomparsave study of removal options performed by the 
contractor. A large part of the EE/CA’s cost is the initial sampling of the site. The 
sampling results subdivide the contaminated area into several sectors based on ordnance 
densities (number of sihanes per acre). Sampling usually involves choosing randomly 
located grids within a sector and surveying them with a magnetometer. Al] anomalies 
are then excavated to determine density. The contractor uses the sampling data to 
perform analysis on removal action alternatives and to develop plans for the Removal 
Action phase. EE/CA costs can be substantial. Current algorithms recommend sampling 
a minimum of two percent of a sector in order to determine ordnance density with a 
reasonable ieprew or confidence [Barrett 1999]. | 


3. Removal Action Phase 


Huntsville Center estimates that Removal Action (RA), an extremely labor- 
intensive operation, constitutes roughly 80% of the CTC [Young 1999]. Most RAs are 
“mag-and-flag” operations. Highly trained ordnance technicians divide the entire area of 


contamination into grids, and then subdivide the grids into approximately 20-foot wide 
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lanes. The technicians sweep the lanes with handheld imaiees and place flags 
wherever an anomaly in the earth’s magnetic field is registered. Finally, the technicians 
use manual methods to dig up all anomalies. A newer method, called geophysical 
surveying, consists of sweeping the entire area with systems that automatically record 
anomaly locations and signatures. The data is then analyzed to filter out false readings 
from actual UXO. Unlike mag-and-flag, geophysical operations only remove those 
anomalies determined to be UXO. Geophysical methods can greatly reduce the cost of 
the RA by reducing the amount of “digs” to perform; however, geophysical surveying 
adds to costs in the EE/CA stage because of the additional resources and expertise 
required in the investigation phase. 

All of the services have Explosive Ordnance Disposal (EOD) units; however, 
large-area environmental restoration differs in many ways from the typical missions 
performed by military EOD units. Due to this and the operational commitments on EOD 
units, civilian contractors perform nearly all Removal Actions on environmental 


restoration projects. 


B. FACTORS AFFECTING COST 


The factors that influence cost are well known throughout the UXO community, 
but the interrelationships of these factors are not fully understood. An economic model 
developed by members of the Naval Explosive elaine Disposal Technical Center in 
the early 1980s, shows the cost of range cleanup to be based on four factors: proposed 
fitaré use, environmental characteristics of the site, expended ordnance characteristics, 


and method of removal [Hersey and Pederson 1981]. 


1] 





1. Proposed Future Use and Depth of Clearance 


Proposed future use is one of the primary factors in the cleanup cost because it 
dictates the depth of removal. Table 1 shows clearance depths prescribed by the DOD for 


varying categories of future land use. 


Projected Future Use —— of Clearance 













SL a 


Table 1: Projected Future Use and Required Depth of Cleanup. The depths listed 
in this table are default depths for ordnance removal. Actual depths may differ in 
accordance with contractor, DOD, and stakeholder negotiations. From [United 
States Department of Defense, Office of the Secretary 1995]. 






As the depth of clearance increases, the technology needed to locate UXO changes and 
the methods to excavate it become more complex. Although not necessarily indicative of 
all projects, the following table of projected costs to cleanup Naval Station Adak, Alaska 


shows exponential cost growth associated with increases in clearance depth. 
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Depth Of Clearance Cost in Dollars/Acre 


Table 2: Estimated Costs for Varying Depth of Clearance on Adak, Alaska. As 
depth of clearance increases, estimated costs per foot increase exponentially. Note 
that a four-foot clearance is approximately ten times the cost of a surface clearance 
while the ten-foot clearance is nearly one thousand times more expensive [Rohrer 
1997]. 


Zi Environmental Characteristics 


The environmental characteristics of a contaminated area including topography, 
vegetation, and soil type also influence cost. The topography of the site may determine 
what methods of surveying and removal to use. Very hilly or mountainous terrain can 
limit both detection and removal to man-portable means only. Similarly, extremes such 
as frozen tundra or swamps limit certain technologies and removal methods. Vegetation 
of the range can also determine cost; often it must be removed prior to surveying and 
ordnance removal. For example, the dense jungle hampers the clearance of Panamanian 
ranges [Panama Canal Treaty Implementation Plan Agency 1997] while even the simple 
shrubs of the northern California coast make the clearance of the former Fort Ord costly 
and difficult [Meuser, and Szasz 1997]. Finally, soil type affects the maximum depth that 
ordnance penetrates, the ability of detectors to locate UXO, and the level of difficulty of 
removing ordnance. For example, ordnance penetrates far deeper into soft sand or silt 
then into hard clay or rock. Similarly, it is far more difficult to locate ordnance in clay 


and rocky soils because the heavy mineral content often produces false anomalies during 
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magnetic surveys. Finally, it is physically more difficult to excavate ordnance in clay and 
rocky soils than in softer soils. 


3. Ordnance Characteristics 


Density (number of UXO per acre) and the types of ordnance present are also 
factors that influence cleanup costs. The density of the ordnance corresponds directly to 
the amount of excavation needed and the overall amount of work to be performed. More 
items in the ground means more holes need to be dug, and more items unearthed means 
more items to treat and dispose of. The poor status of current detection technology 
compounds the costs associated with higher density because the false alarm rate is often 
as high as six false alarms for every actual piece of ordnance [USD (A&T) 1998]. The 
type of ordnance and its delivery method also affects how deep the ordnance penetrates 
the earth. Finally, removing different types of ordnance clearly involves different means 
of removal. DOD ranges contain anything from small grenades a few inches long to 
2,000 pound bombs several feet in length. In particular, chemical weapons require 
specialized removal methods. Figure 1 depicts the myriad of ordnance that ordnance 


technicians encounter in UXO removal. 
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Figure 1: Common Types of UXO and UXO Categories. Different types of UXO 
impose a varying degree of difficulty for their removal. UXO type is thus an 
indicator of the cost of UXO removal. From [USD(A&T),1998]. 


4. Removal Methods 


The cost factor generating the most interest and debate among experts is removal 
methods and removal technologies. As previously discussed, current methods for UXO 
removal are labor intensive manual operations. A Defense Science Board report [United 
States Department of Defense, Office of the Under Secretary of Defense for Acquisition 
and Technology (USD (A&T)) 1998] concludes that approximately $125 million is spent 
yearly by the DOD for UXO cleanup and that $70-$80 million of this cost is for labor. 
The report recommends a coordinated research and development (R&D) program and a 
doubling of the current amount spent on R&D. The Board believes that labor costs can 


be drastically reduced by the implementation of automating technologies. As the report 


States: 
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Despite limitations of the current R&D program, there is enough 
scientific understanding and experimental data to convince the Task Force 

that an aggressive and well managed program could demonstrate dramatic 

improvements in cost effectiveness within the next few years. [USD 

(A&T) 1998] 

The technologies being developed span the complete array of remediation 
requirements. Although the focus is on improving sensor capabilities used in detection, 
other technologies involve computer systems for automatic and digital mapping of a 
contaminated area, robots and advanced excavators for mechanized removal, and 
artificial intelligence knowledge-bases for discriminating UXO anomalies. Some of 


these emerging technologies are slowly being adopted; however, substantiated cost 


relationships based on these technologies do not exist. 
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IV. MODEL DEVELOPMENT 


A. | EXISTING ORDNANCE DATABASES AND MANAGEMENT SYSTEMS 

Huntsville Center contracted Computer Systems Technologies to develop the 
Ordnance and Explosive Database (OED) [Computer Systems Technology 1996], but the 
contract was not fully completed. The database was intended to record historic 
information on all ordnance removal projects, provide a means to analyze information for 
~ use in future cost estimates, and provide a cornerstone for an overall project management 
system. Currently the database is sparsely populated and contains no cost information. 
The OED requires further development to become a mature system capable of fulfilling 
the needs of the USAESCH. 

Besides the OED, two other DOD projects propose to develop an ordnance 
management system that contain information on ail known and potential sites 
contaminated with UXO. A partnership between the Defense Environmental Security 
Corporate Information Management Program Office and the Department of Defense 
Explosive Safety Board is sponsoring the development of the Defense Explosive 
Management Suite which contains a module for ordnance management called 
Unexploded Ordnance Management [Hoehl 1997]. The other proposed system that 
incorporates overall program management is the Range Information Management System 
(RIMS) [Martino et al. 1998]. Although all three systems are in early stages of 
development, these projects suggest that some future system will exist that contains 


detailed ordnance cost information. This thesis designs a cost-risk model to interface 


17 








with the OED, but the theory can be used to design a cost analysis module that can be 
incorporated into any future management system. 


B. DATA DESCRIPTION 


Since the OED is not populated with cost data, the author gathered cost 
information from USAESCH reports and created a test database that attempts to parallel 
the OED design. Final Removal Action Reports, EE/CA Reports, and project manager 
files are the sources for cost information. Currently collecting cost information from 
these sources is a slow manual process, and this limits the amount of cost records 
available for this thesis. Only recently has there been a requirement for the contractors 
to deliver information in a digital form. Most eoamaon exists only in paper reports 
stored in Huntsville Center’s warehouse. 

In relation to UXO program management, Huntsville Center defines a “site” as 
DOD property that contains UXO. A military base, proving ground, or test range is a 
site. A “project” is a subset of a site where some phase of remediation has occurred or is 
occurring. A site can have multiple projects each with a separate project manager. An 
“Ordnance Operable Unit (OOU)” is a subset of a project. An OOU is an ordnance- 
contaminated area that has homogenous physical characteristics and properties. Usually 
the removal action is similar throughout the OOU. Since a project can contain several 
very diverse OOUs, cost data must be captured at the OOU level in order to compare two 
cleanups. This thesis collects the costs for Removal Action of 19 OOUs. These 19 
OOUs comprise 12 different projects from 11 separate sites. Table 3 ree the 


breakdown of the data set. 
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Site Location Number of OOUs per Project 


Plattsburgh AFB, NY 
Miramar NAS, CA 

Camp Croft, SC project 1 
Camp Croft, SC project 2 
Fort Sheridan, IL 

Camp Maxey, TX 

Fort Devan, MA 
Savanna Army Dep, IL 
Fort Wingate, NM 

Erie Ord Depot, OH 
Duck Target Facility, NC 
Fort Ord, CA 








Table 3: Breakdown of Data by Site. The thesis uses cost data for 19 OOUs to 
demonstrate the cost-risk model. The site location and the number of OOUs from 
each project are shown. Camp Croft is the only site used with data available for 
more than one project. The data for each site reflects information that was 
available during data collection, and may not encompass all projects for the site. 
The larger sites such as Fort Ord may contain more ongoing or completed projects, 
but the reports were not found. 

The OED’s design uses a 72-line work breakdown structure (WBS) for reporting costs in 
all phases of remediation. Although this type of detail is desired, there is no requirement 
for the contractors to report costs with this level of fidelity. Until requirements exist that 
force contractors to report costs in this format, the OED’s cost input structure may be an 
unrealistic design. Cost data, however, is routinely reported in “labor,” “material,” and 
“travel/per Diem” costs. This thesis, therefore, collects cost data for each OOU using this 
breakdown. The thesis also assumes that these costs can be converted to costs per acre 
because they are generally variable costs dependent on the length of time to complete a 
project. Travel costs, however, are fixed costs that depend little on the size of the 


cleanup. The travel costs could not be separated from per Diem costs, but in general 


travel is a small fraction of the overall “Travel/per Diem” costs. This breakdown 
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provides enough fidelity for estimates, but the cost-risk model can be adapted to handle 
higher fidelity WBSs. 


C. COST-RISK ANALYSIS 


The majority of the remainder of this chapter provides a general background of 
cost-risk analysis. Readers well versed in cost-risk analysis may skip to Subsection 
IV.C.4. Weighted Estimate of a Distribution Function. Much of the background 
discussion on cost-risk analysis in this chapter summarizes information contained in “The 
| Probabilistic Approach to Cost Analysis” [Book 1999] and “Cost-risk and Cost 
Estimating Uncertainty Guidelines” [Anderson and Cherwonik 1997]. Additional 
sources of background iioaniaiin include many introductory operations analysis 
textbooks and the users’ manuals of commercially available simulation software. Both 
the @Risk [Palisades Corporation, 1997] and Crystal Ball {[Decisioneering Incorporated 
1997] users’ manuals, contain thorough discussion on performing cost-risk analysis. 


1. Problems Associated With Point Estimation 


The current engineering cost models estimate cleanup costs with a single point 
estimate. Due to uncertainty, it 1s highly unlikely that any point estimates are exactly — 
correct, and a single point estimate gives the decision-maker no information on the 
probable range of cost values. Point estimates attempt to provide a single “best 
estimate,” but these point estimates suffer from ambiguity in what the “best estimate” 
really means. Should this “best” estimate represent the “most likely” (mode) cost, the 


“average” (mean) cost or the 50th percentile (median) cost? 
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Often, cost analysts determine total program costs by summing up the “best 
estimate” for each cost element of a program. Cost analysts call the result the “roll-up” 
total cost, and often this “roll-up” is statistically incorrect because each cost element of 
the total cost has uncertainty, and therefore has an underlying probability distribution. 
Furthermore, cost analysts may report some of these “best estimates” as the mean of the 
underlying distribution, some as the mode, and some as the median. Even if the analyst | 
reports each cost element in the same manner (e.g., all modes), summing these values 
may not give the intended results. When the analyst sums measures of central tendency 
of varying probability distributions, very rarely is the result the matching measure of the 
distribution of the total. The reason is because very few probability distributions having 
a mean, mode, and median located at the same value. The triangular distribution’s 
probability density function (PDF) illustrates this point. The peak of a triangular 
distribution is the “most-likely” (mode) cost, and the end points are the extreme low and 
high costs. Figure 2 shows a triangular distribution of a cost element where the mode, 


mean, and median are not collocated. 
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2,000 $2500 $3,354 $3,500 - $6,000 
Low Mode Median Mean High 


Figure 2: The Triangular PDF. The triangular PDF is an example of a distribution 
where the mode, mean and median are not collocated. 


When many independent cost elements are summed to form a total cost, the 
central limit theorem states that the distribution of the sum should have an approximately 
normal distribution with a mean equal to the sum of the individual means. Since the 


mean, median, and mode of a normal distribution are all equal (located at the mid point of 


the distribution) it follows that: 
1. Total cost mean = Sum of cost element means. 
2. Total cost mode = Sum of cost element means. 
3. Total cost median = Sum of cost element means. 


And, therefore, it is most often the case that: 


1. Total cost median 4 Sum of cost element medians. 
2. Total cost mode #4 Sum of cost element modes. 


Often in point-estimates the “best estimate” is the mode estimate, but Figure 3 
shows a common example of when the sum of the modes of the cost element distributions 


does not equal the mode of the total cost. In this case the “roll-up” cost lies to the left of 


the total cost mode. 
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Normal Distribution of the Sum of the Cost-Elements 


Distribution of Total Cost 
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Most likely total cost by Mode, Mean, Median 
roll-up method. Actual most likely cost 


Most likely 


Figure 3: Roll-up Most Likely Cost vs Actual Most Likely Cost. The figure shows 
that the sum of the cost element modes is often not equal to the mode of the total 
cost distribution. 

For most distributions the sum of cost elements’ most likely costs does not equal the most 
likely total-cost. When the number of cost elements is small, there is no expectation that 


the total-cost distribution would be normally distributed. However, it is still unlikely that 


the sum of the most likely costs equals the most likely total cost. 


25 





2. Development of Total Cost Probability Distributions 


In DOD acquisition programs, the roll-up method tends to underestimate costs by 
a wide margin [Book 1999]. The DOD is therefore shifting to a method known as “‘cost- 
risk analysis.” The method involves developing a PDF for the overall cost. Two 
approaches exist for developing the total-cost PDF. The “top down” approach involves 
subjectively determining a PDF for the total cost while the “bottom-up” approach builds 
a PDF for each cost element. The “bottom-up” approach is usually preferred because it is 
often much easier to estimate the PDFs for the individual me elements than for the total 
cost. 

In the “bottom-up” approach the cost analyst estimates the PDF for each cost 
element. For example, it may be very hard for a cost analyst to subjectively determine 
the PDF for the total cost of developing a new fighter anette Therefore, the analyst 
breaks down the aircraft into all of its constituent parts In military acquisition, this is 
usually done in the work breakdown structure. The analyst may be able to make good 
estimates on the PDFs for the development costs of the engine, avionics, weapon system, 
radar, and other constituents. Next, a computer statistically sums the individual PDFs by 
Monte Carlo sampling to develop a PDF for the total cost. The final PDF provides the 
user with a probabilistic estimate, provides information on cost percentiles, and provides 
the means to produce confidence intervals for the estimated cost [Book 1999]. 

The first step in a “bottom-up” cost-risk analysis is to develop the PDFs for each 
cost element. Often in cost estimating very little information is known about the true 


distribution of cost; however, even limited information can aid the cost analyst in 
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estimating a parametric distribution. The following discussion shows the most 
commonly used cost estimating distributions and how basic assumptions about costs can 
be used for choosing a PDF. 

There are three qualities that make a probability distribution desirable for 
modeling cost: the distribution contains only positive, the distribution has a right skew, 
and the distribution has easily estimated parameters. Cost analysts usually desire 
distributions with non-negative ranges because they usually define costs as non-negative 
values. A right skew for cost distributions is a result of market place economics. Costs 
tend to be located around the mode, but very high costs occur with small probabilities. 
Unfortunately for the consumer very low costs do not usually happen with any 
probability. Two commonly used cost distributions that have these properties are the 
Weibull and lognormal distributions. These distributions are usually parameterized with 
two parameters, but an alternate three-parameter distribution can be used to shift the 
distributions away from the origin. These robust distributions can take many shapes and 


can usually be developed to fit right skewed cost data. 
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Cost 


Figure 4: A Right Skewed lognormal Distribution. Both the lognormal distribution 
and certain Weibull distributions can take on the right skewed shape as seen in this 
figure. Both of these distributions are desirable for modeling cost data because their 
ranges are non-negative and the right skew is a typical phenomenon of costs. 
Although the Weibull and the lognormal distributions tend to model true cost 
distributions well, their parameters are not easily defined. In contrast, cost analysts 
sometimes use the triangular distribution because it can be constructed easily. As 
previously introduced, this distribution requires three parameters: an absolute low cost, a 
most likely cost, and an absolute high cost. Even with limited prior data these parameters 


can be determined based on an expert’s opinion. A triangular distribution appears in 


Figure 5. 


Cost 


Low Most Likely High 
Figure 5: Triangular PDF Construction. Cost analysts often use the triangular 
PDF to depict the distribution of a cost-element because it can be easily created 
with only assumptions of the low, high, and most likely cost. 


Cost analysts also commonly use the normal or Gaussian distribution. The 


normal distribution’s familiar symmetric bell-shape supports the assumption that the cost 
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is equally likely to be higher or lower than the most likely cost. The normal distribution 
is a two-parameter distribution that is completely defined by its mean p and its standard 
deviation o. The distribution benefits from the fact that its mean, mode, and median all 
occur at the same point centered in the middle of the distribution. This property 
simplifies many computations, and this point represents the “best estimate” of the cost 
under any definition (mean, mode, or median). To construct the distribution, the analyst 
often estimates the value that is greater than 95% off all possible costs. This value is 
roughly 1.96 standard deviations from the most likely cost, and the analyst uses this fact 
to determine the value of the standard deviation. With this value, the analyst has both 
parameters for defining the distribution. The normal PDF also gains favor from the fact 
that many naturally occurring numerical distributions are well estimated by this PDF. 


Figure 6 depicts the normal PDF. 


Cost 


Mean, Median, Mode 
Figure 6: Normal PDF Construction. Cost analysts often use the normal PDF when 
there is an assumption that the cost is equally likely to be greater than or less than 
the most likely cost. The mean, median, and mode are all located at the same point, 
which simplifies many computations. 

The third distribution that cost analysts sometimes use 1s the uniform distribution. 


This distribution does not have a single most likely value, and all outcomes are equally 


likely to occur between some high and low cost. It is unlikely that the uniform 
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distribution models the true nature of a cost element, and is only used when there is no 
idea about the relative likelihood of possible outcomes. The uniform distribution can be 
modeled with two parameters: low cost, and high cost. Figure 6 depicts the uniform 


PDF. 


Low Cost High Cost 


Figure 7: Uniform PDF Construction. Cost analyst use the uniform PDF when — 
there is no knowledge of the most likely cost, but it is assumed that the costs fall 
within a range bounded by a high and low cost with equal probability. 

In the second step of a “bottom-up” risk analysis, one random sample is taken 
from each of the cost elements and then summed together to form a single sample for the 
total cost distribution. The Monte Carlo simulation performs this sampling numerous 


times to form a frequency distribution of the total cost. Figure 8 graphically depicts this 


procedure. 
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Travel/Per Diem Cost 


Total Cost 


Material Cost 





Figure 8: Monte Carlo Simulation to Form the Total Cost PDF. Monte Carlo 
simulation produces a total cost probability distribution from individual cost 
element distributions. 


3. Sampling From Historic Data 


When an abundance of historic cost data exists, parameterized PDFs do not have 
to be hypothesized. Instead of sampling from the hypothesized PDFs of each cost 
element, samples from the empirical data set can be taken. To demonstrate how this 
method works, assume that : cost analyst uses cost data from the cleanup of 15 OOUs to 
estimate the cost of a proposed cleanup. Figure 9 presents the histogram of the labor 


costs for the 15 selected OOUS. 
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Figure 9: Histogram of Labor Costs for 15 OOUs. Monte Carlo simulation can use 
empirical cost distributions from the cost element instead of hypothesized PDFs. 


The method of sampling from historic data is analogous to creating a spinner that 
has slices proportional to the frequency of each value. For the above histogram the 


spinner looks like Figure 10. 





Sampling Labor Costs 
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Figure 10: An Analogy for Sampling From Empirical Data. A spinner models 
sampling from empirical data by creating slices proportional in size to the relative 


frequency of each cost. 
By this analogy a person draws a random sample by spinning the spinner and recording 
the value under the spinner’s pointer. A cost structure composed of N cost elements 


requires N separate spinners. For each sample the person spins the N spinners, records all | 
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N values, and sums the values to form one sample for the total cost. The person performs 
this procedure many times to form a histogram of the total cost. 


4. Weighted Estimate of a Distribution Function 


Both methods of sampling, from historic data and from hypothesized PDFs, have 
advantages and disadvantages. It may be difficult to fit cost probabilities with a common 
parameterized distribution, while sampling from historic information does not require 
fitting a distribution. Sampling from historic data provides some actual information 
about the true cost distribution. However, sampling from the historic data suffers when 
only a few data points exist because sampling purely from these points is equivalent to 
assuming that the costs can only take on one of these few values. Most likely the data set 
does not contain the values near the extreme high or extreme low of the true cost 
distribution. Thus, when sampling purely from the historic data, a lot of information in 
the tails of the distribution may be. The ideal probability distribution for a cost element 
benefits from the properties of both a hypothesized parametric probability distribution 
and an empirical data set. 

The benefits of both methods of sampling can be obtained by using a method 
similar to the Bayesian estimate of the eipution function [Ferguson 1973, p.272]. 
Ferguson shows that the Bayesian estimate of a probability function can be expressed as 
the weighted average of the hypothesized prior distribution and the empirical distribution 


of the data. For a data set of nm observations the Bayesian estimate is 


FIX eX) = PhO+tA- pF GNX nX) (4.1) 


Where: 
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Pp, =~AlA+n (4.2) 
—fo(t) - The prior hypothesized PDF 

f(t |_X,,....X,) - The empirical PDF given the data set Xi,...,Xn 

a- Weighting factor 


Equation (4.1) weights the prior distribution by , and the empirical distribution © 
by (1-9, ). It can be seen that as m increase p, decreases; hence, the equation weights the 
prior distribution less and weights the empirical distribution more. The constant a is a 
weighting factor that can be interpreted as the faith in the prior distribution measured in n 
units. This thesis uses this weighted distribution to estimate cost element probability 
distributions. 


5. Handling Correlation 


The traditional Monte Carlo sampling theory of summing cost elements to 
produce a total cost probability distribution is valid only when the cost elements are 
independent. It is often the case that cost elements are not independent but correlated. 
Two random variables X and Y are correlated when the value of X has some predictive 
power on the value of Y. For example labor costs and material costs in UXO cleanups 
have an intuitive relationship that leads to correlation. A difficult cleanup that uses a lot 
of materials requires a lot of labor hours to employ the materials; hence, a strong positive 
correlation exists. Traditional Monte Carlo sampling provides the correct value for the 
total cost mean regardless of whether the cost elements are independent or correlated; 
however, the variance of the total cost probability distribution depends strongly on the 


correlation of the individual cost elements [Book 1999}. Let X), X2, ...X, be n random 
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variables representing m cost elements, 4; the mean of cost element Ss o; the standard 


deviation of cost element X; and TC be the total cost. It follows that: 


Total Cost: TC = > x; (4.3) 
i=] 
Mean of Total Cost = ps X, } YU; (4.4) 
i=] t=] 
Variance of Total Cost = var » xX, | (4.5) 
i=] 
n n jrl 
2 
= » So; © 2) > PFS; (4.6) 
i=l j=2 i=l 


As seen in equation (4.4) no cost element correlation term appears in the 
calculation of the total cost mean; however, the correlation coefficient, pj, appears in 
equation (4.6), the calculation of the total cost variance. Not accounting for correlation is 
equivalent to setting p;; equal to zero, and equation (4.6) then simplifies to: 


r=] 


1 2 
Var of Total Cost = »y 0; (4.7) 


The differences in equation (4.7) and (4.6) shows that strong positive correlation 
can greatly increase the total cost variance while strong negative correlation can greatly 
reduce the total cost variance. Thus not accounting for correlation gives the incorrect 


total cost probability distribution. Accounting for correlation is not a simple proposition: 
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The ideal way to simulate total cost would be to specify inter-cost- 
element correlations, generate correlated random numbers, and sum them. 
‘Probability theory allows this to be done exactly only for multivariate 
normal cost distributions, not in general [Book 1999] 

Book further points out that Lurie and Goldberg [1998] have been working for 
several years on the correct method to generate correlated random numbers that correctly 
model cost-element cost distributions. 

An accepted approximate solution to the above problem is to use rank-order 
correlation, a distribution free measure of correlation. Several rank-order measures of 
correlation exist including Kendall’s tau and Spearman’s rho. The two most popular 
software packages that perform cost-risk analysis: Crystal Ball [Decisioneering 
Incorporated, 1998] and @ Risk [Palisades Corporation 1997], use rank-order correlation. 
Rank-order correlation between a pair of data sets involves ranking the corresponding 
sets from lowest to highest value. To illustrate the calculation of Spearman’s rho, Table 


4 shows the labor cost per acre and the corresponding material cost per acre for 10 


OOUs. Next to each cost is the rank of the cost within its column. 
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Lo | = $360.20 20 

ee 
3 | $7521 | 8 
_ 4 | $80914 | 4 
6 | $1,196.95 | 6 
_ 7 | $1,402.56 | 7 


8 
9 | $3,068.89 
| 10 =| $4,225.71 


Table 4: Determining Spearman’s Rho by Rank Ordering Cost Elements. This 
example shows how to determine rank-order correlation between variable X (Labor 
cost) and variable Y (Material Cost). 
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If the cost elements have perfect positive rank-correlation (9 = +1) then the 
corresponding rank value for each pair would be equal. If the cost elements have perfect 
negative correlation (p = -1), then the lowest labor costs would be paired with the highest 
material costs, and the highest labor costs would be paired with the lowest material costs. 


Equation (4.8) calculates p [Conover 1999, p. 315]: 
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. n+ 1 
_ 2 R(X .)R(Y,)- 27 : (4.8) 
I 1 
n 2\2f _n 2 \2 
Bb: roxy a AS4) Vy pony a( 254) 
i=] 2 i=] 2 
Where: 
R(X;) — Rank of the ith X variable 
R(Y;) — Rank of the ith Y variable 
For the above example (X = labor cost, Y = material cost, n = 10), p is calculated 
as follows: 
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p= 338 = 302.9 = 439 (4.9) 


(385 —302.5)2 (385 —302.5)2 

The process of generating rank-correlated pairs of sampled values is a two step 
process that requires the user to supply correlation coefficients for each pair of data. In 
the first step, the simulation software generates randomly distributed rank scores for each 
random variable. The software generates one score for each iteration of the simulation. | 
Sampling 10,000 times would require 10,000 random rank scores for each random 
variable. These rank scores are then rearranged to give pairs of scores that match the 
desired rank correlation coefficient. In the second step, the software generates another 
set of random numbers for each random variable, and again generates one number for 
each iteration of the simulation. For each variable the procedure assigns the smallest 
random number to the smallest rank score, the second smallest random number to the 
second smallest rank score, and so on. The final result is a set of pairs of random 
numbers that can be used in sampling values from correlated distributions [@Risk User’s 
Manual, p.179]. The software creates correlated pairs for aes random variable with 


every other random variable and finally arranges the pairs to fit a matrix of specified 


correlation coefficients. 
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V. MODEL APPLICATION 


This thesis creates a probability distribution of the CTC of a proposed 

_ contaminated area by analyzing cost data from previous cleanups. Location 
characteristics, ordnance characteristics, and cleanup requirements can be vastly different 
for two separate projects. In order to account for variance in costs caused by these 
differences, the cost-risk model uses only data from areas “similar” to the proposed area 
of cleanup. Once similar records are found the sosinsk model uses the historic cost data 
to generate cost per acre estimates that incorporate cost-risk analysis. 

As previously mentioned, USAESC, Huntsville maintains the Ordnance and 
Explosives Database of completed and ongoing UXO removal projects. Although this 
database is sparsely populated with cost information, there is interest within the DOD to 
produce and maintain a DOD-wide ordnance and explosive management system. The 
assumption of this thesis is that either the OED will be populated in the future or another — 
repository of information will be created. This thesis presents a model with designs to 
incorporate the OED data fields, but the model can be easily modified to accept historic 
information from any repository of UXO data. Appendix A outlines the information 
fields needed by the cost-risk model but that currently do not exist in the OED. The same 
appendix also makes recommendations to modify some fields so that the cost-risk model 


can incorporate them. 
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A. DATA FIELDS 


-The following discussion outlines the necessary fields that the cost-risk model 
must obtain from a database. Although these recommendations have designs to 
specifically incorporate the OED, the same recommendations hold for any other UXO 
data repository if developed. The cost-risk model uses database fields segregated into 
three groups: record identification, OOU descriptive dione and cost information. 
The record identification fields provide a means to identify each OOU as unique. The 
fields include site, project, and OOU identifiers. Table 5 shows the required record 


identification fields. 


FicddName_|________ Descriptions 


Name of site (e.g., Fort Ord, Aberdeen Proving Grounds). | 
project that an OOU falls under. 

Unique identifier within each project. Identifies a specific 
OOU within a project. ; 


Table 5: Record Identification Fields. The cost-risk model needs record 
identification fields to uniquely identify OOUs. All fields exist in the OED. 







.The second group of fields is the OOU descriptive characteristics. These fields 
include environmental characteristics, ordnance characteristics, and cleanup 
characteristics. These are the fields that the user queries to find records similar to the 
proposed cleanup. Table 6 summarizes the OOU descriptive characteristic fields required 


by the cost-risk model. 
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Table 6: Ordnance Operable Unit (OOU) Descriptive Characteristics. The cost- 
risk model requires these fields to identify an OOUs environmental characteristics, 


its unexploded ordnance characteristics, and its cleanup characteristics. 
*Field currently exists in OED 

’Recommend modification of existing field in OED (see Appendix A) 

“Recommend addition of field in the OED (see Appendix A) 


The final required fields from the database are the cost information fields. The 
cost fields in the database summarize information on six cost elements that make up total 


cost. These elements are: © 


. EECA Labor Cost per acre. 

. EECA Material Cost per acre. 

EECA Travel/Per Diem Cost per acre. 

. Removal Action Labor Costs per acre. 

. Removal Action Material Cost per acre. 

. Removal Action Travel/Per Diem per acre. 


An hWN 


The model converts all costs to FY99 costs. The cost-risk model produces estimates 
following a completed ASR; Herefare it does not include cost information on the first _ 
two phases of the remediation process (Preliminary Assessment and Investigation (ASR) 
phases). The cost fields that the cost-risk model obtain from the database differ from the 


fields in the current OED structure. The recommended fields, however, reflect data that 
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is realistically accessible. Ideally cost data would exist for all phases of a project 
(Preliminary Assessment, Investigation, and Removal Action); unfortunately this thesis is 
only able to acquire Removal Action cost data. Huntsville Center cost engineers estimate 
Removal Action costs account for 80% of the total cost to complete [Young 1999]. For 
the purpose of demonstrating this model, the thesis assumes the costs for the three EECA 
cost elements to be equal to two percent of its corresponding RA cost element. The basis 
of this rough figure is the current procedure of sampling approximately two percent of an 
OOU during the EECA phase in order to determine ordnance density. Table 7 outlines 


the cost fields that the cost-risk model uses. 


Field Name | Description 






RA LABOR COST? RA Labor Cost for the OOU 
RA MAT COST RA Material Cost for the OOU 
RA _TRAVPDIEM_COST? RA Travel and Per Diem cost for OOU | 





Table 7: Ordnance Operable Unit (OOU) Cost Information. These fields are 


needed to report cost information. 

*Field currently exists in OED 

*Recommend modification of existing field in OED (see Appendix A) 
“Recommend addition of field to OED (see Appendix A) 


B. FINDING SIMILAR CLEANUPS 


The user imports the current ordnance data file into the cost-risk model, and then 
begins developing a cost-risk estimate. If the ASR is complete for a site, the user has 
approximate information on the areas to be cleaned up. The user’s first step is to find 


records similar to a proposed area of cleanup by filtering the data based on the OO0U 
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descriptive characteristics. An important trade-off exists when filtering; as the user 
specifies more criteria the amount of available records decreases. Conversely, the more 
detailed the search criteria the more similar the selected OOUS are to the proposed 
cleanup area. Unfortunately a detailed search may leave few records for sampling. Since 
the current data set is small the user is only able to filter the data using a few different 
criteria. As the database grows in number of cleanups, the user can increase the search 
criteria. 
C. FILTERING PRECEDENCE 

The user filters the records beginning with the OOU descriptive field that is most 
indicative of cost. If after a query enough records remain, the user refines the search by 
querying on the next most influential field and so on. The user stops filtering when he 
determines that further filtering could reduce the data set below a useable size. The OO0OU 
descriptive fields can be subdivided into three groups based on how strongly they | 
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influence cost. These groups consist of “type of cleanup,” “primary indicators,” and 
“secondary indicators.” 


1. Type of Cleanup 


The TYPE field determines if the cleanup is a chemical weapons cleanup or a 
conventional weapons cleanup. Users should always filter this field first since chemical 
weapon cleanups are far more difficult than conventional weapons, and costs of chemical 


weapons cleanups are not indicative of the costs of conventional cleanups. 
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2. Primary Indicators 


Plotting each OOUs descriptive characteristic versus the total cost for RA 
determines the remaining filtering precedence. Primary fields are those fields dias 
indicate a strong relationship between them and costs. The first plot shows density of 
each OOU plotted against RA total cost. Although there are several possible measures to 
describe the density of ordnance within.an OOU, the cost-risk model uses total scrap 
(UXO scrap and non-UXO scrap) as the measure of density. Density usually refers to 
the amount of UXO per acre, but this definition may not be : good indicator of costs. 
Ordnance technicians must remove UXO scrap as well as non-UXO scrap from a 
contaminated area, and they still must excavate false anomalies caused by non-UXO 
along with actual UXO. The relationship pcarecs actual UXO density and non-UXO 
scrap density and its effect on costs is extremely difficult to define. This thesis defines 
density as the total scrap metal removed from an area. A more detailed analysis of the 
relationship of UXO scrap and non-UXO scrap and its effects sieust iS bexond the scope 
of this thesis and should be the subject of further research (see Appendix A for further 
discussion). Figure 11 shows the relationship of density to cost. This plot shows a strong 


relationship of increasing costs with increasing density. 
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| Figure 11: Scatter Plot of OOU Ordnance Density vs Total Cost. The scatterplot 
shows a strong relationship of increasing RA costs with increasing density. The log 
trend line suggests that costs increase proportionally to the log of density. 





Figure 12 shows that the increase in cost is roughly linear with the log of density. 
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Figure 12: Scatter Plot of OOU Ordnance Log-density vs Total Cost. The figure 
shows a strong relationship of increasing RA costs with increasing log-density. The 
- trend line shows that the relationship is approximately linear. 





The relationship of depth of clearance to cost appears weaker than that between 


density and cost, but the plot shows that as depth of clearance increases total cost tends to 
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increase. Unfortunately the data set does not contain any clearances in excess of four 
feet. Figure 13 plots of depth of clearance versus cost and adds a linear trend line to 


show the increasing relationship. 
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Figure 13: Scatter Plot of OOU Depth of Clearance vs Total Cost. The figure shows 
a relationship of increasing RA costs with increasing depth of clearance. 


A similar relationship appears in the plot of vegetation difficulty factor versus 
total cost (see Appendix A for discussion of difficulty factors). In Figure 14 the reader 


sees positive correlation between total cost and the vegetation factor of an OOU. As the 


difficulty factor increases the cost of ordnance removal tends to increase. 
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Figure 14: Scatter Plot of OOU Vegetation Difficulty Factor vs Total Cost. The 
figure shows a relationship of increasing RA costs with increasing vegetation 
difficulty factor. 


2. Secondary Indicators 


Appendix B contains the scatter plots of the remaining OOU characteristics: 
topography, soil type, and ordnance type. These plots do not indicate the expected cost 
relationships. The author hypothesizes that these secondary fields do not provide the 
expected cost relationships because the primary indicators overshadow the effects of the 
secondary indicators. For example, clay soil is difficult because clay contains a lot of 
minerals that make detection of UXO difficult using conventional magnetometers. Most 
eens assumes that if two cleanups are identical except for soil type, the area with the 
higher soil type difficulty factor will be more expensive. Unfortunately, the only data 
points with clay soils also have low densities and low vegetation factors, and thus have 
low costs. The second explanation for why secondary indicator cost relationships do not 


reflect expected results is interactions with the primary fields. For example, one can 
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hypothesize that heavily used firing areas tend to be located in flatter terrain, or that 
certain soil types indicate the amount of sustainable overgrowth. Analysis with 
additional data should better reveal the true cost relationships of these factors. 

D. MODEL INTRODUCTION 

1. Initial Worksheets 

Visual Basic for Applications creates the cost-risk model as an add-in for 
Microsoft Excel. When the user loads the add-in into Excel, one worksheet is visible and 
two worksheets are hidden. The hidden worksheet “InflationIndices” contains the 
inflation indices needed to normalize all costs to base year 1999. The other hidden 
worksheet is “Tables” which provides non-changing descriptive information about the 
OOU characteristics. The user should not need to alter or access sihiseor these 
worksheets. 

When the model user opens the add-in the “Master” worksheet is the only visible 
worksheet. This worksheet contains all of the Record Identification, OOU Descriptive 
Characteristics, and OOU Cost Information fields, as discussed in subsection V.A.2. The 
add-in can update these fields from a text file of current data and automatically convert 
costs to fiscal year 1997 costs. Figure 15 shows the cost-risk model when initially 


opened. 
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Figure 15: View of Cost-Risk Model on Opening. This figure shows the only visible 
worksheet when the Excel add-in is loaded. Visible in the figure is the OOU number 
and associated OOU characteristics. The model updates this worksheet from a text 


file of current data. 
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Additionally, the add-in contains several forms that the user can use to input 


filtering criteria. The scenario discussion, later in this thesis, presents each of these forms 


in detail. These forms provide a Graphical User Interface and simplify navigation 


through the model. Figure 16 shows one of the forms that assists the user in filtering 


records. This form is the Depth of Removal Form used to filter the records based on a 


minimum and maximum depth of clearance. 
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Depth of Removal 





Figure 16: Example of a Form. The Depth of Removal Form is an example of the 
many forms available to assist the user in entering data. The user can use the scroll 
bars at the bottom to set the minimum and maximum depth of clearance. The 
buttons in the upper right help to navigate through the model. 


3. Menus 


The add-in creates two additional menus in Excel: “Cost Forms” and 


“Cost_Analysis.” These two additional menus appear on the right of the toolbar. 





Figure 17: Added Excel Menus. The cost analysis add-in creates two additional 
menus in excel. These menus are “CostForms” and “Cost_Analysis,”’ and are 
visible on the far right of the menu bar. 


The Cost Forms menu provides additional means to navigate through the different 


forms; this menu lists all available forms. Figure 18 shows this pull down menu. 
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Figure 18: The CostForms Menu. The CostForms menu assists the user in. 
navigating through the available forms. 


The “Cost_Analysis” pull down menu contains submenus that allows the user to access 
the model’s important functions. These three functions are: 1) Filter the records to find 
similar cleanups, 2) Perform an analysis on the filtered historic data, and 3) Build 
probability density functions for input into the simulation software. The scenario 


presentation discusses each function in detail, and Figure 19 shows the “Cost_Analysis” 


menu. é 


ISTE SAT Re 





Figure 19: The “Cost_Analysis” Menu. The “‘Cost_Analysis” menu provides access 
to the model’s important functionality including filtering records, analyzing historic 
costs, and building probability density functions. 
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E. SCENARIO DEVELOPMENT AND MODEL DEMONSTRATION 


This thesis uses the following scenario to demonstrate how to interface with the 


cost-risk model and to aid the user in the understanding of the internal workings of the 


model. 


Scenario: The Marine Corps conducts an Archive Search Report on 
Marine Corps Base Camp Lejeune, North Carolina. The ASR shows a 
suspected Ordnance Operable Unit of 45 acres contaminated with a 
mixture of high explosive ordnance including 155mm Artillery rounds, 
81mm and 60mm mortar rounds, 40mm grenades, and hand grenades. 
The ASR does not suspect any small arms ammunition. The area has not 
been used for several years and the vegetation is overgrown with shrubs 
and small trees. The contaminated site is in the western part of the base 
with predominately flat terrain. The soil type is a mixture of silt and sand. 
EOD units conduct semiannual range sweeps of this area, and only a small 
amount of surface scrap is found during the ASR. Further, range control 
records show that range usage for this area is low because of its proximity 
to the edge of the base property. The Marine Corps plans to convert this 
area back to a maneuver area. 


1. Step One: Find Similar Cleanups by Filtering Database 


Using the “Cost_Analysis” menu the user selects the sub menu “GUI Search” 
(Graphical User Interface Search) or “Manual Search.” The GUI Search submenu starts a 
sequence of user forms that help refine the search criteria. The Manual Search submenu 
sends the user to the Master worksheet where the user can search the records using 
Excel’s AutoFilter function. 


a. Filter by Type of Cleanup 


If the user selects “GUI Search” the model first presents the Type of 
Cleanup Form where the user checks what type of cleanup concerns them: conventional 


ordnance and explosives or chemical weapons. The ASR for the scenario shows it is a 
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conventional cleanup, and therefore the user selects “Ordnance and Explosives” as Figure 


20 shows. 


opener one ee ee ee ee cpa ea 


Click the type of Cleanup 





© Ordnance and Explosives 
© Chemical | 


Figure 20: The Type of Cleanup Form. This form helps to filter the database based 
on the type of cleanup. When the user selects the Ordnance and Explosives Button 
the model filters out all of the chemical weapons cleanups. 
By selecting the “Next>>” Button the model presents the user with the 
next filtering form; the Close and Navigate buttons close out the form and allow the user 
to go to different forms or worksheets. Finally, the Filter Button shows the user how 
many records are remaining in the filtered set. By selecting “Ordnance and Explosives” 
and clicking the Filter Button, the user reduces the data set to nineteen records (i.e., the 


current data base does not contain any chemical weapons cleanups). Figure 21 shows 


the Found Records Form that displays the number of remaining records. 
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Click the type of Cleanup 


Found Records 


Your Filter Criteria has reduced the | 


number of similar cleanups to: - 





Figure 21: The Found Records Form. The user can use the Found Records Form 
from any location in the model to show how many records remain in the selected 


data set. 
b. Filter by Density 


The “Next>>” Button sends the user to the Density Form. Density tends 
to be a strong predictor of cost, unfortunately prior to surveying the user may only be | 
able to make a rough guess on the eepecied density. The user should use a broad range 
if there is uncertainty concerning the density. The form provides four preset ranges: 
“Sparse,” “Low,” “Medium,” and “High.” Dividing the current data set into 
approximately equal quartiles, determines these preset ranges. For the given scenario, the 
ASR shows that the Marine Corps did conduct periodic range sweeps and that 
investigators found only a small amount of surface scrap during the site investigation. 
For these reasons the user 1s somewhat confident that the range is sparsely populated and 


selects a liberal density (total scrap per acre) range of 0-100 pounds per acre. This 
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selection reduces the data set to 14 records. 


22: 


vhageasgnasns yi 
ny Near laes 





The Density Form. The Density Form allows the user to filter the data 
set based on the total scrap. Prior to performing an EECA the user may only have a 
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VEGETATION Difficulty Factor 


1: Fit hansen or lowgyass. = 


Table 8: Vegetation Characteristics and Associated Difficulty Factors. In 
accordance with the RACER/ORCA model, the cost-risk model ranks vegetation by 
difficulty factors. Unlike density, the user has certainty of the type of vegetation _ 
contained in a contaminated area. 








The contaminated area in the scenario contains heavy vegetation; therefore, the user may 
choose to select categories V4 and V5. This selection narrows the data set down to 11 


records. Figure 23 is a depiction of the Vegetation Form. 


Query Records Based On Vegetation Type @ 


egetation is ranked in difficulty by the following categories: 
V1: Flat Barren or Low Grass ) 

V2: Low Grass and Few Shrubs 

¥3: Heavy Grass with Numerous Shrubs 

V4: Shrubs with Some Trees 


V5: Heavy Shrubs and Trees or Forest 


Query the records by choesing one vegetation type or several vegetation types. 





Figure 23: The Vegetation Form. The model user can use this form to set the 
minimum and maximum vegetation type for the contaminated area. For the 
current scenario a range from V4 to V5 best depicts the area. 
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d. Filter by Depth of Clearance 


Following the Vegetation Form is the Depth of Removal Form. The final 
depth of clearance requires deliberation among all concerned parties prior to RA, but the 
intended future use of the land suggests about a four-foot clearance. Setting the 
minimum depth to two feet and the maximum depth to six feet reduces the number of: 


available records to nine. The Depth of Removal Form appears in the Figure 24. 


Fae karan vr api ht he Seen tee tee et etna et eeet bie ere ere Liha 


[Depth of Removal. 8 0 eB 





Figure 24: The Depth of Removal Form. The Depth of Removal form allows the 
user to find historic cleanups with similar clearance depths. For the current 
scenario, filtering by depth reduces the amount of available records to nine. 

e. Filter by Topography 

Once the user completes filtering on the primary cost indicators he begins 


filtering with the secondary cost indicators. Due to small number of records in the 


filtered data set (9 records), the user would likely quit at this point. When more records 


a) 








are added to the database, the user would likely have enough remaining records to 
continue refining the search. However, in this case the user should concentrate on 
removing the extremes of the secondary indicators so as not to significantly reduce the 
number of records any further. Like vegetation the model ranks topography using the 
RACER/ORCA difficulty factors. Table 9 lists the topography categories and the 
associated difficulty factors. The cost-risk model labels these categories as T1 through 


Td: 






a eee a 









Table 9: Topography Categories and Associated Difficulty Factors. In 
Accordance with the RACER/ORCA model, the cost-risk model ranks topography 
by difficulty factors. 

The contaminated area in the scenario falls in the T1 and T2 categories, by selecting a 
minimum of T1 and a maximum of T4 the user eliminates one record in the mountainous 


category (T5) leaving eight records in the data set. Figure 25 shows the selection of 


topography categories. 
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Query Records Based On Topography Type a me - Next? 3 


opography is ranked in difficulty by the following categories: 
Ti: Flat Terrain 
T2: Gently Rolling Terrain 
T3: Heavy Rolling Terrain or Flat Terrain with Gorges 
T4: Rolling Terrain with Gorges 
: Mountainous Terrain 


Query the records by choosing one topography type or several topography eae 
2 ora els Min Topographry | Topography 


Max Topography 





Figure 25: The Topography Form. The Topography Form allows the user to filter 
the data set based on the predominant topography of the OOU. This figure shows a 
minimum topography of T1 and a maximum topography of T4 selected. 


f. Filter by Soil Type 


The next form is the Soil Type Form (Figure 26). Soil categories have 
ORCA/RACER difficulty factors (Table 10) and the cost-risk models labels these 


categories as S1 through S6. 


Di 


Si: Sand 
/S5:_Clay/Sand with stone | 
[S7: Rock 8 


Table 10: Soil Categories and Associated Difficulty Factors. In accordance with the | 
RACER/ORCA model, the cost-risk model ranks soil types by difficulty factors. 











The scenario site is in the $2 to $3 categories, and since the user does not want to 
eliminate too many records he selects $1 through SS to eliminate the very difficult 
cleanups in the clay and rock categories. Using these criteria the filtering removes no 


further records. 
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Soil is ranked by increasing difficulty in the following categories. 


$1: Sand 
: Gravel Sand mixture 
Silt or Sand-Silt 
Clay/Sand or Clay Silt 
Clay/Sand with stone 
Clay 
Rock 


Query the records by choosing one soil type or several soil types. 





Figure 26: The Soil Type Form. The Soil Type Form allows the user to filter the 
data set based on the OOU’s predominate soil type. This figure shows a minimum 
soil type of S1 and a maximum soil type of S4. 


g. Filter by Ordnance Type 


The final filtering category is ordnance. Here again, difficulty factors for 
ordnance types exist as specified in RACER/ORCA. Table 11 lists ordnance types 


ranked by difficulty factor. 
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Ordnance Categor Difficulty Factor 
Ol: Bombs (Practice) 


O7: Bombs (Explosive) 





Table 11: Ordnance Categories and Associated Difficulty Factors. The cost-risk 
model ranks ordnance RACER/ORCA difficulty factors. 


Often there is uncertainty in the amount and type of ordnance that is present in an OOU. 
In some cases a range contains only one specific type of ordnance, at other times the 
contaminated area has everything from practice rounds to high explosive submunitions to 
2000-pound air delivered bombs. In order to filter OOUs by ordnance type, each OOU 
in the database has a mean ordnance difficulty factor. Equation 5.1 salesiliai the 
ordnance difficulty factor by a weighted average of the proportion of an ordnance type 
and its difficulty factor: 

> Expected Proportion ,, * Difficulty Factor 5, (5.1) 

ia OT is the ordnance type 

| In many eases it is difficult for the user to estimate the type of ordnance 
and proportions of each type of ordnance; however, there is usually enough information 
to rule out extreme records. In the eceuatia the ordnance difficulty factor has a value 
equal to a minimum of (0.4). Ruling out records below (0.3) is a viable solution. Figure 


27 shows the Ordnance Form. 
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The ordnance difficulty factor is based on the weighted average of the 
difficulty of each type of ordnance in the contaminated area. Launch the 
Ordnance Difficulty Calculator to determine the difficulty factor of the 
contaminated area. Use this value as a reference to set minimum and 
maximum ordnance values. 


“WNour Calculated Ordnance Btw 
= Difficulty Factor is: 


Query the records by choosing an ordnance difficulty range. 





Figure 27: The Ordnance Form. The Ordnance Form allows the user to filter a 
range of ordnance types based on their difficulty factor. The figure shows the 
minimum factor set at 0.4 and the maximum factor set at 1. 

To help decide the ordnance difficulty factor for the. contaminated area, 
the user can click the Calculate ODF Button in the center of the form. This button 


launches the “Ordnance Difficulty Calculator” to determine the mean difficulty factor 


using the inputs and equation 5.1. Figure 28 is the Ordnance Difficulty Calculator. 
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Ordnance ranked by increasing difficulty . 


Practice Bombs, Grenades, Landmines 

Detonators, Blasting Caps, Fuzes, Boosters, Bursters 
Explosive Grenades (Hand and Rifle} 

Small Arms 

Explosive Landmines 

Explosive Rockets, Guided Missiles 

Explosive Bombs, Medium/Large Caliber 


ee Petes 
= - 


Percent 06 in Contaminated Area 
wPercent 07 in Contaminated Area 


mathe tt “ tia Pa gt fhe ee. 





Figure 28: The Ordnance Difficulty Calculator Form. To help the user determine 
the average ordnance difficulty factor, the user checks the types of ordnance and 
fills in the percentages of that type. The user assumes that the area contains 34% 
03, 33% O06, and 33% O7. The bottom value (0.73) is the mean difficulty factor. 


Once the usér 1s satisfied with the filter criteria the values can be 


reviewed by selecting the Values Form. 


62 











_Shrubs with some trees 


_Heavy shrubs and trees or forest 





“Max Topography ~ ; | _Roling with Gorges: 


Figure 29: The Values Form. The user can view the Values Form at anytime to see 
the filtering criteria. 


Z Step Two: Historical Analysis 


With the data set reduced down to a satisfactory group of similar cleanups, the 
user selects “Historical Analysis” from the “Cost_Analysis” pull-down menu. This 
selection generates three additional sheets in the workbook. These sheets are 
“DescStats,” “Histograms,” and “Correlation.” 


a. Descriptive Statistics of Similar Cleanups 


The “DescStats” sheet is the descriptive statistics on each cost element of 


the filtered set. The cost-risk model presents the minimum, median, average, maximum, 
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and standard deviation of the data set in a matrix. Table 12 is the output of “DescStats” 


for the scenario cleanup. 


|___Per Acre Costs | Minimum | Median | Mean | _Max_| Stan Dev_ 

















$1,210] $1,840] $2,571] $7,520] $2,218 








Table 12: Output of Historical Analysis “DescStats” Worksheet. This table | 
appears in the worksheet “DescStats” after running a historical analysis on the 
filtered set of similar cleanups. The worksheet contains descriptive statistics of the 
individual cost elements. 


b. Histograms 


The second sheet contains the Histograms for each of the six cost 
elements. The histograms give a graphical representation of what has been spent in a 


specific area for similar cleanups. Figure 30 is a view of this worksheet. 
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Figure 30: Output of Historical Analysis Histograms. This figure shows the second 
sheet that the model generates when performing a historical analysis. Visible in the 


figure are two out of the six cost element histograms. The user can use this visual 
information to get a feel for the empirical distribution and make inferences on the 


true distribution 
Cc. Correlation Matrix 


The final sheet produced during the historic data analysis is 
“Correlations.” This sheet contains the correlation matrix from the filtered data set. The 
matrix shows how each cost element correlates with every other cost element. The 


correlation matrix for the unfiltered data set and the scenario data set appear in Table 13. 
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All Records EECA Labor EECA EECA RA Labor RA Material RA Travel 
Material Travel | PerDiem 
PerDiem 
uel Ae a A eee 
EECA | 0.710 710 ] | 1.000 
Material 


Travel 

a 
Riad a cH a 0.799 | 1.000 | #| |. 
0.714 6 lU6U6l 0.908 0.714 1.000 | 


RA Travel | 0.799 0.906 0.800 0.907 1.000 
Rites | 0759] 0900 — St ee | eee | 


Filtered Set | EECA Labor EECA EECA RA Labor RA Material RA Travel 
Material © Travel PerDiem 
——- 


— 
sor | 
Material 


Tar cam ~ t=} 
Travel 
on . | 
anne Baal 0.075 0.258 | 1.000 {| £| | 
0.082 | Hie 0.938 0.080 1.000 |  _—s 


RA Travel 0.257 0.932 0.263 0.937 1.000 . 
nc 2k 


Table 13: Correlation Matrices for all Records and the Scenario Records. The 
upper matrix is the correlation matrix that the model generates when performing an 
historical analysis on all the records in the database. The Lower matrix is the 
correlation matrix for the scenario data set. The off diagonal correlation 
coefficients that equal 1.0 (highlighted values) result from how the EECA data was 
generated. 











The values highlighted in the table have a correlation value of 1.0 because taking 2% of 
the corresponding RA values generates the unavailable EECA data. One should note the 
high correlation in the unfiltered set for RA data. Material and Labor have a value of 
0.71, Labor and Travel/per Diem have a value of 0.8, and Material and Travel/per Diem 
have a value of 0.90. For the filtered set these values were, 0.08, 0.263, and .937 


respectively. The coefficients in the filtered set tend to be much smaller except for 
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Material-Travel/per Diem. The small correlation in the filtered set is likely a result of a 
few outliers in the small set. This thesis recommends that the user input correlation 
coefficients closer to the unfiltered set. As data sets of similar cleanups become larger 
the user can choose the correlation coefficients from the filtered data set. 

3. Step Three: Create Cost Element Probability Distributions 


a. Choosing Alpha 


The next step prior to simulation is to create weighted probability density 
functions for each cost element (the sakea model uses a linear combination of the 
empirical distribution and the user’s supplied triangular distribution to form a weighted- 
distribution). The user must determine the weighting factor @ for each cost element 
distribution (see Equation 4.1). The weighting factor a expresses the user’s faith in the 
triangular distribution. For example when @is equal to n (the number of records) the 
empirical distribution is equal in weight to the triangular distribution. When @ equals 
zero the formula gives no weight to the triangular distribution and when @is very large 
the weight shifts to the triangular distribution. Figure 31 visually shows the effects of 
varying @. The first graph Figure 31 is the histogram of the weighted PDF with alpha 
equal to zero. The second histogram is with an alpha equal to the sidan of records (19). 


The final histogram uses an alpha of 100. 
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Figure 31: Increasing Alpha and the Shape of the Weighted PDF. This example uses 
the same empirical distribution and triangular distribution but varies the alpha. 
With o = 0 the first histogram mirrors the empirical PDF. With an o = n the middle 
PDF is slightly triangular with spikes at the discrete values. At o = 100 the last PDF 
is nearly triangular with some slight spikes. 


b. Inputting Triangle Parameters 


By selecting “Create Cost Distributions” from the “Cost_Analysis” menu, 
the cost-risk model presents the user with the “Set Triangular PDFs” form. Visible on 
this form are the statistics for the cost element (minimum, mean, average, maximum), 
and input boxes to set alpha and the triangular distribution’s parambters (minimum value, 
most-likely value, and maximum values). The model sets the default value for a equal 
to the number of records, it sets the triangle’s minimum and maximum to 80% of the 
empirical distribution’s minimum and 120% of the empirical distribution’s maximum, 
and sets the triangle’s most-likely value equal to the empirical distribution’s mean value. 
The default values are only starting points and may not be realistic or desirable. The user 
should carefully consider each cost element when setting a and the triangular 


distribution’s parameters. 
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Set Tangle. PDFs 00. Se ee BR ke a es 


Cost Element Name: 


$2,500.00 : 





Figure 32: The Set Triangle PDFs Form. The user uses this form to set a, and set 
the triangular distribution’s parameters (minimum value, most-likely value, and 
maximum value). The form shows the statistics for each cost element and the user 
can click on the Show Histogram Button to view the histogram of the filtered data. 
The form displays the name of the cost element in the center, and the user uses the scroll 
bar next to the cost element’s name to cycle through all of the cost elements. For the 
scenario, the form shows that seven similar cleanup records are found. For the Removal 
Action labor costs the user may feel that the hypothesized triangular distribution is as 
valuable as the empirical distribution, and sets alpha equal to seven (the number of 
records). An alpha equal to the number of records gives equal weighting to the empirical 


distribution and the traingular distribution. The user looks at the statistics along the left 


side of the form and then reviews the histogram for the cost element (Figure 33). 
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Figure 33: The Histogram for Removal Action Labor Costs in Dollars/Acre. The 
histogram for RA labor costs does not reveal much of a trend in the data, but 
provides a visual depiction of costs of similar cleanups. 

Assume that a cost analyst believes that contractors never charge less than $2,500/acre 
and that the government never enters into contracts for this type of cleanup for more than 
$8,000/acre. To provide more insight the user runs the RACER/ORCA model and gets a 
point estimate of $6,200/acre for the labor costs . The analyst builds a triangular | 
distribution based on the RACER/ORCA estimate and the historic data. The user sets the 
minimum of the triangular distribution at $2,500/acre, the center at $5 000/acre, and the 
maximum at $8,000/acre. The user then enters these values at the bottom of the form. 
The same methodology applies for all cost elements. When the user gets to EE/CA 
material costs he feels that the data accurately reflects the true distribution (unlikely with 
only seven records) and decides to put all of the weight onto the empirical data by setting 


a for this cost element to zero. Conversely, policy changes have altered how per Diem is 


paid and the user determines that the empirical data is too high and not reflective of 
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current costs. The analyst thus sets a for “RA Travel/Per Diem” cost to 100, heavily 
weighting the user supplied triangular distribution. 

When the user clicks the Done button the model creates the weighted 
PDFs for each cost element composed of 10,000 random points from the empirical and 
triangular distribution. The cost element’s & determines how many points come from 
each distribution. If the weighting is 4/5 for the empirical and 1/5 for the triangular 
distribution then the model generates 8,000 random points from the empirical distribution 
and 2,000 points from the triangular distribution. Code is written in visual basic using 
the inverse transform method to create these distributions. Any simulation package can 
then draw with equal probability from these 10,000 numbers to form the weighted- 
distribution. | 

4. Step Four: Run Simulation 


a. “QutPutReport”’ 


The cost-risk model creates two additional sheets at this time: 
“OutPutReport” and “SimInputs.” The “OutPutReport” sheet summarizes the statistical 
data for each cost element, the triangular distributions for each cost element, the filtering 


criteria, and the correlation matrix. Figure 34 shows the information that the 


“OutPutReport” sheet contains. 
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Figure 34: The “OutPutReport’ Worksheet. The “OutPutReport’ worksheet 
shows the statistics for each cost element, the triangular distribution parameters, & 
for each cost element, the filter criteria, and the correlation matrix for the found set. 
The correlation matrix is not visible in the figure. 


b. “SunInputs”’ 


The worksheet “SimInputs” (See Figure 35) contains formulas composed 
of the random variables, and the simulation software uses this sheet to run the simulation. 
The worksheet’s formulation works with @ Risk but can be easily modified to interface 


with other simulation packages. 
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Figure 35: The “SimInputs” Worksheet. The “SimInputs” worksheet sets up the 
simulation. Cells D3 to D8 draw random samples from the weighted PDFs of the 
corresponding cost elements. Cell D11 is the sum of all RA only cost elements, and 
D12 is the sum of all cost elements. 

Although different simulation packages require slightly different 
formulation, in general the inputs are the cost element PDFs and the outputs are fankeons 
of these inputs. A great advantage of simulation is that multiple functions of the inputs 
can be created. The “SimInput” sheet provides an example of this with one output being 
the sum of all cost elements and one being the sum of only the RA inputs. In this way 
one derives an RA total cost per acre PDF and a combined RA/EECA total cost per acre 
PDF (both PDFs are in dollars/acre). Further the user could write another function that 
sums up the inputs and then multiplies them by the acreage of the contaminated area. 
This final formulation becomes the total cost PDF in dollars. After the user writes any 
additional output formulas he desires, he enter the inputs and outputs into @Risk along 


with the correlation matrix and then runs the simulation. 


F. JUSTIFICATION OF FILTERING 


To demonstrate that the filtering can accurately locate cleanups with similar costs, 


the thesis runs two additional scenarios. The thesis considers the original scenario as a 
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medium difficulty cleanup because it contains a mixture of difficult and easy OOU 
characteristics. The density is low to medium; the vegetation is high in difficulty; the 
depth of clearance and ordnance type is medium in difficulty; and the terrain and soil 
types are easy. The “hard” and “easy” scenarios contain the same amount of records (7) 
as the original, “medium,” scenario. The “hard” scenario filters the records by setting all 
the OOU characteristics in the difficult ranges (i.e., high density, high vegetation 
difficulty factor, deep depth of clearance, etc.). The “easy” scenario does the converse, 
setting the OOU characteristics to the lower degrees of difficulty. For all cost elements 
and for all summary statistics, the “easy” cleanup has lower values than the “medium” 
cleanup and the “medium” cleanup has lower values than the “difficult” cleanup. Figure 
36 shows the minimum, asin average, and median value for each cost element of 


each scenario. 
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Figure 36: Summary Statistics for “Easy,” “Medium,” and “Hard,” Scenario. The 
model develops three scenarios: “Easy” cleanup, “Medium” cleanup and “Hard” 
cleanup. The filtering process finds similar cleanups as can be seen in the summary 
statistics. For each cost element the minimum cost, the maximum cost, the median 
cost, and average cost increase as difficulty increases. 
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VI. SIMULATION OUTPUT AND INTERPRETATION 


A. SINGLE CLEANUP RESULTS 


The Monte Carlo simulation is run for the Chapter V scenario cleanup using the 
“all records” correlation matrix at the top of Table 18, and by setting @Risk for 10,000 
iterations. To emphasize the necessity of considering correlation effects, the thesis shows 
the results for an identical simulation that assumes zero correlation among cost elements. 
Appendix C contains the output report from the scenario. Figure 37 shows how Monte 


Carlo sampling forms a total cost PDF from the cost element PDFs. 
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Figure 37: The Total Cost PDF for the Scenario. Monte Carlo Simulation samples 
from the individual cost element PDFs to produce the total cost PDF for the 
‘‘medium” scenario presented in Chapter V. The simulation is run with 10,000 
iterations and the “all records” correlation matrix. All values are in dollars/acre. 
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The @Risk software package generates the total cost PDF shown at the bottom of the 
previous figure. Since only a few cost elements exist in the simulation, the total cost PDF 
does not approach the normal distribution shape, but the distribution does begin to 
smooth out. 

To portray the importance of considering correlation, Figure 38 compares the 
uncorrelated (left side) and correlated (11 ght side) total cost probability density functions. 
As is expected, the PDF for the uncorrelated case is more symmetrical and less spread out 


(less variance). 
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Figure 38: Comparison of Uncorrelated and Correlated Total Cost Per Acre PDFs. 
The comparison of the uncorrelated (left side) and correlated (right side) simulation 
outputs shows that uncorrelated random variables produce a tighter distribution. 
Costs are in dollars/acre.. 
The comparison shows that by ignoring correlation, the analyst assumes an artificial 
“reduction in risk.” Reduction in risk refers to the fact that there is a reduced chance of 
getting values in the tail 

In addition to the PDF, the simulation software generates the summary statistics 


for the total cost distribution. Table 14 lists the summary statistics for the correlated and 


uncorrelated scenario cleanup. 
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Table 14: Correlated and Uncorrelated Summary Statistics for the Scenario 
Cleanup. The simulation software produces summary Statistics for all input and 
output cells. This table lists the summary statistics for the total cost per acre of 
performing an EECA and RA on the contaminated area. 














The summary statistics for the correlated inputs show that for the scenario cleanup the 
simulation estimates total cost per acre to fall between $4,239/acre and $25,694/acre. 
The results estimate the most-likely value to be around $8,400/acre (mode), but shows in 
the long run an average around $11,715/acre (mean). As is expected, both the correlated 
and uncorrelated scenarios have the same mean but the standard deviation is much 
greater for the correlated case. 

Before the cost analyst can make informed decisions about costs, he needs the 
cumulative probability distribution for the total cost. The cumulative probability 
distribution is also an output of the simulation software, and Table 15 shows tabulated 


values of the cumulative probability distribution. 
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X X Probability that True 
(Correlated) | (Uncorrelated) | Total Cost is Less than X 


$6,141 $8,272 





| 
| $1924] $12,067] 60 
$18,172] $15,008} 90 





Table 15: The Total Cost Per Acre Cumulative Distribution for the Chapter V 
Scenario. The simulation software generates the table of the cumulative 
distribution. The value in the far right column gives the probability that the total 
cost per acre is less than the corresponding value in the left column. To emphasize 
the effects of correlated random variables, the table gives values for both the 
correlated simulation and the uncorrelated simulation. 

The cumulative distribution shows the probability that the cost of cleanup will fall below 
a given value. As mentioned previously, there is an artificial reduction in the risk when 
not considering correlation. For example the cost analyst may be asked, “what is the 
probability of costs exceeding $15,000 per acre ($675,000 total for 45 acres)?” Using 


the false assumption of no correlation of cost elements, the analyst’s answer is about 


10%. The proper correlated result shows an answer closer to 20%. 
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With the above information the cost analyst can make informed decisions about 
the estimated cost and the risks associated with it. For example, the analyst may feel that 
that $8,600/acre (mode) is the most likely cost for the EE/CA and RA, but he is also 
aware that the probability of exceeding this cost is roughly 15%. For budgeting purposes 
the analyst may choose a value closer to $11,000/acre (median), because the probability 
is equally likely that the cost is greater than or less than this value. 

B. MULTIPLE CLEANUPS | 

Although users of the ree model need estimates for individual OOUs, they 
more often desire project level estimates. Since projects consist of one or more OOUs, 
the user can use the individual OOU estimates to produce project estimates. In an 
individual OOU estimate each iteration of the Monte Carlo simulation forms an input for 
the OOUs total cost probability distribution, and the model maintains these values after 
the simulation. For example, setting the simulation to 10,000 iterations forms 10,000 
random observations from the OOUs total cost distribution. The simulation software can . 
sum the individual total cost probability distributions for several OOUs to form a single 
estuniie for the combined cleanup. To demonstrate this capability, consider a cleanup 
project consisting of three OOUs: the “hard,” “medium,” and “easy” scenario cleanups 
presented in Section V. The “hard” cleanup is 10 acres in size, the “medium” is 45 acres, 
and the “easy” cleanup is 60 acres. To estimate the entire project cost, the analyst first 
produces estimates for each individual OOU. Next, Monte Carlo simulation sums the 
three OOU total cost distributions. Each iteration of the simulation consists of randomly 


drawing a value from each of the three total cost distributions and summing them 
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together. The analyst converts the units of each input distribution from “dollars per acre” 
to “dollars” so that the resulting distribution of the simulation is the total “dollar cost” of 


cleaning the three OOUs. Figure 39 visually depicts the simulation. 
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Figure 39: A “Multi-OOU” Estimate. Users most often desire a combined estimate 
for several cleanups. The user initially produces individual OOU distributions and 
then uses the OOU distributions as the inputs for the “multi-OOU” estimate. This 
figure shows a project that consists of three OOUs: 60-acre “easy” cleanup, 45-acre 
“medium” cleanup, and 10-acre “hard” cleanup. 

VII. The “three-OOU” distribution shows that the most likely cost for cleanup of 
the three OOUs is roughly $1.7 million. Appendix D contains the simulation 
summary for the “three-OOU” estimate. The summary includes the cumulative 

distribtuion and the summary statistics. 
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RECOMMENDATIONS AND CONCLUSIONS 


This thesis demonstrates a model for conducting cost-risk analysis of UXO 
cleanups. This research shows that UXO contaminated areas with similar characteristics 
have similar costs. Given appropriate data the model is capable of finding historic 
cleanups that have characteristics similar to a proposed cleanup. With the historic 
records, the cost analyst has descriptive information to assist him in building cost element 
PDFs. Using the output of Monte Carlo simulation of cost element PDFs, the cost 
analyst has the tools to make a cost estimate that incorporates cost-risk analysis. 


A. RECOMMENDATIONS 
1. Model Advantages and Disadvantages 


The reader should consider the cost-risk model as a prototype. The main shortfall 
of the model is the limited amount of historic data records it contains. This lack of 
records reduces the users ability to find a satisfactory pool of similar cleanups. The 
USAESCH is implementing procedures that will make historic cost data more available 
and thus allow for the population of a cost database. The cost-risk model requires some 
modification to the OED in order to obtain the information in the proper format. 
Appendix A outlines required OED modifications. 

The second shortfall of the cost-risk model is in the cost elements that the model 
uses. These cost elements reflect the cost information format that is available, and they 
do not represent an ideal breakdown of costs. The cost elements have high correlation, 
and because of their broad nature may be difficult for an analyst to estimate. The 


advantage of a large numbe of cost elements is that smaller “pieces” are usually more 
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intuitive for an analyst to estimate. For example “Labor” costs may be more difficult to 
estimate than the components: “Supervisor Labor,” “Ordnance Technician Labor,” 
“Engineering Labor,” and “Quality Control Technician Labor” costs. Appendix E. 
contains the 72-line work breakdown structure (WBS) that is the basis for reporting costs 
in the OED. This thesis recommends that future cost models or modifications to this 
model attempt to use cost elements that are a compromise between the cost-risk model 
cost elements and the WBS costs. Regardless of the cost reporting fidelity, the 
USAESCH should establish a consistent digital cost reporting format that can be entered 
into a database. Currently, contractors submit most cost information in “hard-copy” 
reports. Cost analysts must enter this information manually into the OED. The cost-risk 
model design allows for easy modification to accept additional cost elements. 

One advantage of the model is that it refines itself as more data becomes 
available. Experts in this field believe that as detection technology improves, ordnance 
removal costs will drastically decrease. Future records added to the database would 
reflect these lower costs. Conversely, a parametric cost relationship built today may 
become obsolete due to these cost changes. More records also mean that the set of 
similar cleanups will be larger. 

Another advantage of the model is that its design allows for records to be updated 
from a text file. Maintainers of ordnance information could aeeiens the text file from 
the OED or another ordnance management system and then disseminate it to users around 
the country. The Defense Environmental Network and Information Exchange (DENIX) 


is the ideal means for disseminating this data. This internet service provides DoD 
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personnel access to environmental legislative, compliance, restoration, cleanup, and DoD 
guidance information. It serves as a central electronic "meeting place” where information 
can be exchanged among environmental professionals worldwide. Users of the cost-risk 
model could easily retrieve updated data from this exchange while the DOD could still 
control who is permitted access to the data. 

Finally, the model’s “Historical Analysis” function provides the USAESCH a 
means to analyze what was done in the past. Even if the user is not performing a cost 
estimate, this feature can be used to answer many questions. For example the user could 
manually select records from a specific contractor to analyze whether or not their 
“Labor” costs differ significantly from other contractors. 


2. Further Research 


This thesis gives insight to many other possible areas of research. Primarily, — 
information management research needs to be done to determine how cost information 
can be easily acquired and managed. The OED is intended to gather cost information, 
but it appears that little structure was developed to make this possible. Further, raw cost 
information provides little information without a means to analyze it. The cost-risk 
model’s “Historical Analysis” function provides some analysis, but these features can be 
greatly increased. 

Validation of this model is also needed. USAESCH is currently trying to validate 
the RACER/ORCA models so that USAECH can delegate RACER/ORCA’s use down to 


the individual Regions of the Army Corps of Engineers. This task proves difficult 
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because little information has been kept on initial government estimates and actual costs 
[Young 1999]. 
Finally, the RACER/ORCA models could be easily modified to perform cost-risk 

analysis. According to an USAESCH cost analyst, “These models give accurate results 
as long as the correct assumptions are made [Young 1999].” Unfortunately the models 
have no means for a __ to express uncertainty in their assumptions. The 
RACER/ORCA models could be modified so that all inputs containing uncertainty have a 
probability distribution. For example, a user eal specify the density of the ordnance, 
but this is often uncertain even after sampling. Instead of entering the ordnance density 
as a single value, the user could enter a probability distribution of the ordnance density. 
The final result would be a “bottom-up” engineering model that benefits from cost-risk 
analysis. These chanzes are feasible since the RACER/ORCA models and computer 
code are property of the DOD. Modifying RACER/ORCA may prove easier to 
implement than the cost-risk model presented in this thesis. These models already 
contain extensive features for project management and are already familiar to many users 
who deal with environmental restoration. 


B. CONCLUSIONS 


The Range Rule requires the services to dedicate a significant amount of 
additional resources to an already monumental problem. Only a few agencies such as 
USAESCH have experience in this area, but the cost-risk model gives many possible 
users the ability to use USAESCH historic data to budget and estimate the cost of 


ordnance remediations. Estimating the costs of environmental restoration of ordnance 
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contaminated sites is a difficult process containing many unknowns. The probabilistic 


cost model of this thesis provides the means to manage this uncertainty and risk. 
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APPENDIX A. ADDITIONS AND MODIFICATIONS TO THE ORDNANCE 
| AND EXPLOSIVES DATABASE 


This appendix makes recommendations for additions and modifications to the 
data fields contained in the Ordnance and Explosives Database (OED). These 
recommendations allow for the cost-risk model proposed in this thesis to use the OED 
structure. The OED is an attempt by the United States Army Engineering and Support 
Center, Huntsville (USAESCH) to create a repository of information on unexploded 
ordnance cleanups. The database is not complete and is not-sufficiently populated with 
existing information. The database may likely be the framework for a future unexploded 
ordnance management system, and therefore the thesis attempts to use the current 
structure as much as possible. Further OED developments can incorporate these 
changes. This appendix breaks down the data fields into.two groups: OOU descriptive 
characteristics, and cost information. 

A. OOU DESCRIPTIVE CHARACTERISTICS 

1. Soil 

The OED only allows for three descriptive characteristics of soil type. The OED 
should expand the allowable inputs in order to reflect the structure of the ORCA/RACER 
cost model. The ORCA/RACER model assigns difficulty factors to each soil type, and 
the cost-risk model adopts theses rankings. Table 16 recommends the allowable soil 


inputs and associated difficulty factors. 
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Difficulty Factor 





Table 16: Recommended Soil Categories and Associated Difficulty Factors. This 
thesis recommends that the OED incorporate the above soil categroies and the 

associated difficulty factors. This cost-risk model uses the above structure. From 
[BTG Incorporated (Delta Research Division). Remedial Action Cost Engineering 


and Requirements (RACER) System _3.2.a, Computer Software.1998] 

2. Vegetation 

The OED does not allow enough descriptive categories for the vegetation of an 
OOU. Similar to soil type, the current OED fields do not provide the ability to capture 
the difficulty in remediation associated with differing soil types. Again, this thesis 
recommends the adoption of the RACER/ORCA difficulty factors. Table 17 lists the 


vegetation categories and the difficulty factors. 


[Flatbarrenorlowgrass | 


Table 17: Recommended Vegetation Categories and Associated Difficulty Factors. 
The thesis recommends that the OED incorporate the above vegetation categories 
and associated difficulty factors. This cost-risk model uses the above structure. 
From [BTG Incorporated (Delta Research Division). Remedial Action Cost 


Engineering and Requirements (RACER) System _3.2.a. Computer Software 1998] 
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3 Topography 
The OED only allows three inputs for terrain: “flat,” “rolling,” and “rugged.” In 
order to capture the difficulty in cleanup due to topography this field should use the six _ 


topography categories used in RACER/ORCA. Table 18 lists these categories and their 


associated difficulty factors. 


TOPOGRAPHY Difficulty Factor 





Table 18: Recommended Topography Categories and Associated Difficulty Factors. 
This thesis recommends that the OED incorporate the topography categories and 
associated difficulty factors. From [BTG Incorporated (Delta Research Division). 


Remedial Action Cost Engineering and Requirements (RACER) System_3.2.a. 
Computer Software 1998] 

4. Density 

Density usually refers to the amount of UXO per acre, but this definition may not | 
be a good indicator of costs. Technicians must remove both UXO scrap and non-UXO 
scrap from a contaminated area, and false anomalies must be excavated along with actual 
UXO. The relationship between UXO density and non-UXO scrap density and its effect 
on costs is extremely difficult to define. The cost-risk model records density in total 
scrap per acre. A more detailed analysis of the relationship of UXO scrap and non-UXO 


scrap and its effects on cost is beyond the scope of this thesis and should be the subject of 


further research. 
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>. Depth of Clearance 

‘The OED records depth at which individual UXO is located. There, however, is 
not a field in the OED that captures the contracted depth of clearance. The OED should. 
eeiponils a depth of clearance field. 

6. Type of Ordnance 

The OED and RACER/ORCA use the same inputs for the type of ordnance found 
in the OOU, and these inputs should remain unchanged. The cost-risk model uses the 
RACER/ORCA ordnance difficulty factors. However, since many ranges are multi-use, a 
large variety of ordnance may exist. The OED should represent the ordnance difficulty 
by a weighted average of each type of ordnance and its difficulty factor. Table 19 shows 


the allowable ordnance inputs and their difficulty factors. 





Table 19: Recommended Ordnance Categories and Associated Difficulty Factors. 
The OED already incorporates the above ordnance types but not the 
RACER/ORCA difficulty factors. From [BTG Incorporated (Delta Research 


Division). Remedial Action Cost Engineering and Requirements (RACER) System 


3.2.a. Computer Software 1998] 
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7. Type of Operation 

The “type of operation” refers to the survey and removal methods used at the site. 
Traditional removals use mag-and-flag techniques. More modern methods use global 
positioning surveying and automated recording and mapping. The OED needs to track 
the type of removal as a predictor of costs. Although beyond the scope of this thesis, this 
data will also be important in the future for justifying investment strategies. The OED 
should use the RACER/ORCA categories; Table 20 shows these fields. The OED may 
need to increase the categories if some of the proposed technologies prove useable 


(ground penetrating radar, synthetic dog noses, acoustic, etc.). 


TYPE_OPERATION MEANING 


HH GPS Hand held GPS used. Manual Recording 


GPS Auto GPS surveying with automated removal and 
mapping 


Table 20: Recommended Type of Operation Categories and Their Meanings. The 
OED should track the type of cleanup operation performed on an OOU. The table 
lists the recommended inputs for this field. From [BTG Incorporated (Delta 


Research Division). Remedial Action Cost Engineering and Requirements 
(RACER) System_3.2.a. Computer Software 1998] 








B. COST INFORMATION 

Huntsville Center requested that the OED have a 72-item work breakdown 
structure for reporting cost information. The contractor responsible for populating the 
OED stated that finding cost information in this detail was unrealistic since no 
requirement exists for contractors to report cost information in this detail [Chandler 
1999]. A more detailed cost structure is desirable, but this thesis requires the following 


fields at a minimum. 
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1. Date Fields for Cleanup Phases 

The database needs to add date fields so that cost information can be normalized 
for inflation. The database needs to use year that the phase (Preliminary Assessment, 
Archive Search Report, Engineering Evaluation/Cost Analysis, and Removal Action) was 
completed. 

2. Preliminary Assessment and Archive Search Report Total Cost 

Preliminary Assessments and Archive Search Reports usually have fixed prices. 
These costs usually represent less than 20% of the cost to complete and therefore require 
a single total cost entry. 

3: Labor, Material, and Travel/per Diem Costs for EE/CA and RA 

The majority of the cost to complete occurs in the EE/CA and RA phases. The 
OED, at a minimum, needs to report costs for these two phases by “labor,” “material,” 


and “travel/per Diem” costs. 


96 











APPENDIX B. SCATTER PLOTS OF SECONDARY INDICATORS OF COST 


This appendix contains the scatter plots of the OOU characteristics that the thesis 
considers as secondary indicators of cost. These plots do not show any discernable 
relationship between the OOU descriptive characteristics and cost of ordnance removal. 
The “secondary indicators” are topography, ordnance type, and soil type. Figure 40 


shows the plots of the “secondary indicators.” 
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Figure 40: Scatter Plots of Secondary Cost Indicators. The secondary cost 
indicators include soil, topography, and ordnance difficulty factors. These cost 
indicators do not show expected cost correlations. The causes of these unexpected 
results may be due to interactions with and overshadowing by the primary 
indicators. 
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SCENARIO OUTPUT REPORT 


APPENDIX C. 


Re OE 


SdB1I0H 
WIM 
SUIT[OY ey Aydveasodoy, 
9u0}s 
qm p 
ueg/AeiD purg addy, [10g 


re a 
}SOIOJ JO 
$001) pue 


sqn.iys $99.1} OUIOS 
AAvoH WIM sqniys 01383939 A 


VIda LID NOLLOATAS 
00095 [00S | OOTeS [CLOTS [OOS] _OOSTS | wOTeSS [_aeVTES [RR oTaORT, 


_oos'is | ooo'rs | ooo'ss | oozes | oss | ooss | ooc'ers | asr'scs | sonway ajsuerty 


pose | oe foe foo foo foe |e foe | ay 

















: UOIeIAIG 
SIT T$ OOLI$ CO I$ 8PE$ OSPr$ LETS S7L$ ILp$ prepueys 


cess | ove'ss | oov'os | ours | oes'is | ove'is | oos'ers | oor'ecs 





L 
| ora'is | ooo'is | orr'vs | ozs | oses | ooss_| oou'zrs | oor'ocs 
| ois | oes | cures | 09s | oases | osos | ovs'irs | osu'ses 















S07) Iesjlad | aaesiad | 3a1e/sod 3S07) 3.19¥/19d [270], 
WIIigiag | jsod je 3807) ys0x WIIGI9Ig 3s07) yioday 3807) [230], 
JOABLT, Wajyeyyy | = 10qe’T] jeliajey ELUZR § 10QV’y] yoievas yUIUISSISSY 


Vu 


Vu vu Areuyulypaid 


VOud VOuR | Voda sAIOIY 


Output Report For the Scenario Cleanup. The report shows the 
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APPENDIX D. SIMULATION SUMMARY FOR THE THREE-OOU CLEANUP 


$716,691 







Mean= | _$1,712,636_| 
a eee 
10% Pere = 
25% Perc = 
55% Perc = 
85% Perc = 
95% Perc = 


Table 22: Simulation Summary for the Three-OOU Cleanup. This table shows the 
summary statistics and the cumulative density function for the cost of the three- 
-OOU cleanup. The distribution is nearly symmetrical with the mean, median, and 
mode all roughly $1.7 million. 
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APPENDIX E. 72-ITEM WORK BREAKDOWN STRUCTURE 


Work Breakdown Description 
Structure ID ? 


32 Program Management 
321 Project Management 
322 

3221 
322102 
322103 
322105 
322111 
322112 
322113 
322114 
322115 
322116 
3222 
3223 
322302 
322303 
322304 | 
322305 
322311 
322312 
322313 
322314 
322315 
322316 
3224 
322403 
322404 
322406 
322411 
322412 
322413 
322414 
322415 
322416 
322417 
3225 
322502 
322503 
322504 
322505 
322511 
322512 
322513 
322514 
322515 


In-House Work-Agreements 
Project Support Product Services 


Field Investigations 

Formulation and Evaluation of RA Alternatives 
raft Investigation Report 

Final Investigation Report 

Value Engineering Studies 

Decision Documents and Related Products 


AE and Professional Services Procurement 
AE and Professional Services Contracts 
Project Support Products/Services 

Work Plans 

Records Research 

Field Investigations 

Formulation and Evaluation of RA Alternatives 
Draft Investigation Report 

Final Investigation Report 

Value Engineering Studies 

Decision Documents and Related Products 


AE Supervision and Review 

Project Support Products 

Work Plans 

Field Investigations 

Final Risk Assessment Report 
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Table 23: The 72-Item Work Breakdown Structure. The OED cost reporting 
structure incorporates this work breakdown structure. The structure is 
hierarchical and indents subtasks of primary tasks. Populating the OED with cost 
data proves difficult because contractors rarely report data with this fidelity. This 
thesis uses a more aggregated cost breakdown based on available cost data. 
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